CppCast - API & ABI Versioning

Episode Date: February 16, 2018

Rob and Jason are joined by Mathieu Ropert to discuss C++ API & ABI compatibility issues and Modern CMake. Mathieu is a french C++ expert with an eclectic background. He's worked in various fields including kernels, virtualization, web development, databases, REST microservices, build systems and package management, all those in (or about) C or C++. He is presently awaiting his next challenge in the video game industry that should come up next May in Stockholm, Sweden. Until then, Mathieu lives and works in Paris, France where he is also host of the C++ French User Group. News C++ in 2018 Inspector - A drop-anywhere C++ REPL The 15 C++11 features you must really use in your project Mathieu Ropert @MatRopert Links CppCon 2017: Mathieu Ropert "API & ABI Versioning..." CppCon 2017: Mathieu Ropert "Using Modern CMake Patterns to Enforce a Good Modular Design" Meeting C++ 2017: Mathieu Ropert "The need for a package manager interface" The Ultimate Guide to Modern CMake Bincrafters Blog Sponsors think-cell software Embo++ Hosts @robwirving @lefticus

Episode 138 of CppCast with guest Mathieu Roper, recorded February 14th, 2018. CppCast is sponsored by ThinkCell Software. They're looking for developers that would want to work on extremely challenging C++ development tasks. In their Berlin-based office, they work with no deadlines, no overtime, and no meetings. If this is for you, go to thinkcell.com slash cppcast and apply today. CppCast is also sponsored by Embo++. The upcoming conference will be held in Bochum, Germany from March 9th to 11th. Meet other embedded systems developers working on microcontrollers, alternative kernels, and highly customizable zero-cost library designs.

Get your ticket today at embo.io. In this episode, we talk about what to look forward to with C++ in 2018. Then we talk to Mathieu Roper, host of the French C++ user group. Mathieu talks to us about API and ABI compatibility and modern ZMake. Welcome to episode 138 of CppCast, the only podcast for C++ developers by C++ developers. I'm your host, Rob Irving, joined by my co-host, Jason Turner. Jason, how are you doing today? I'm doing pretty good. Feeling a little frazzled with a bunch of uh travel that i've got planned coming up but i won't go into the details at the moment but anything you want to share or just get a busy

year out of you a busy year at the moment there might be some things to share in the near future but nothing at the moment okay okay well keep us updated uh actually i did want to congratulate you you hit was it 102 episodes of c++ weekly recently uh yes you went past 100 and i forgot to mention that my hundred and tooth if you will was uh was monday awesome congratulations on that thanks okay well at the top of our episode, I'd like to read a piece of feedback. This week, we got an email from Florian, and he was writing about the recent episode with Jonathan Mueller. You were a great show. I regularly listen on my commute.

I just listened to the episode with Jonathan Mueller. It's been 10 years since he completed his computer science diploma at Aachen University. And some comments brought him down memory lane. And he says he does think university should teach, he does not think that university should teach programming. They should teach concepts and the ability to abstract the problem in general. But I guess that was a topic we discussed on the show and how we were both surprised that they weren't teaching much actual language in Jonathan's coursework. Well, yeah. And to be fair, my university curriculum is actually highly theoretical also.

We only had one class on programming, but they also didn't require us to use any particular programming language or toolkit or whatever after that. We were able to pretty much do what we wanted to do. I did some projects in C++, some in Java at the time, whatever. Okay. I guess I had a different experience because I was actually a computer engineer, so I did a whole lot of electrical engineering stuff and then some comp sci

stuff. Right. Okay, well, we'd love to hear your thoughts about the show. You can always reach out to us on Facebook, Twitter, or email us at feedback at cpcast.com. And don't forget to leave us a review on iTunes. Joining us today is Mathieu Roper. Mathieu is a French C++ expert with an eclectic background. He's worked in various fields, including kernels, virtualization, web development, databases, REST microservices, build systems, and package management.

All of those in or about C or C++. He's presently awaiting his next challenge in the video game industry. That should come up next May in Stockholm, Sweden. Until then, Mathieu lives and works in Paris, France, where he is also the host of the C++ French user group. Welcome to the show. Hey, welcome, guys. Thank you very much for having me.

I'm curious what's going to happen to the Paris user group after you move to Stockholm. Oh, well, technically speaking, I only started to give some people a hand. Basically, what came up is that at the time, the host, Joël Falcoux, which you probably know from one talk or ten, had a very busy calendar with his work.

So I just happened to be there at the time of my company, wanted to be the host. So I just helped people to do that. And, well, things leading to another. And now I, yeah, I guess I don't even have to ask him before. I can just, I guess I don't even have to ask him before. We have a pretty tight C++ community that we built up at my company in the past couple of years. And I'm pretty sure I will be able to find somebody else. You already know Jonathan, who I think you interviewed in the past.

But I think we have also two other people who were also speakers at conferences or present suppose i guess one of them would be able to type to step up and uh and do that after me and joel's actually been on the show also yeah so we're just rounding out the leaders of the french c++ community at the moment i guess so tell us a little bit though about your opportunity in stockholm coming up uh yeah uh basically I'm going to Paradox Development Studio. I don't know if you know those guys. They make strategy games like Cruiser Kings, Europa Universalis, and recently they made Stellaris, which is less historic than the previous one, but still about strategy stuff.

Okay. Yeah, I think I've seen them in Humble Bundle, Strategy Bundle kind of things. Yeah, I think it was maybe last month. Okay. Okay, I think I've seen them in like humble bundle, strategy bundle kind of things. Yeah, I think it was maybe last month. Okay. Okay, interesting. Well, Matthew, we've got a couple articles to discuss. Feel free to comment on any of these, and then we'll start talking to you more about some of your recent conference talks, okay?

Sure. Okay. So this first one is a post from Jens Veller on meeting C++. He was just talking about C++ in 2018, kind of what was accomplished in the community last year and what we have to expect over the course of this year. One interesting note I thought he had here was talking about Meltdown Inspector, which is a topic we've been talking a bit about recently. And he kind of considers it to be a plus in a way for the c++ community because it kind of shines a light on performance

which you know is always very important to c++ programmers but we kind of have a leg up in how we can handle the performance tax imposed by meltdown inspector compared to you know if you're a javascript programmer and this affects you there's not really a whole lot you can do about it. Yeah, the whole world just got slower. May as well program in the language that lets you have a handle on why, I guess. Exactly.

If you're working in something like high-frequency trading where that impact really affects you, hopefully using some of the workarounds that have been developed, you can get around it and get back to the same level of performance. Right. Go ahead.

No, I'm sorry. I was just going to comment that this comprehensive list of the conferences was bigger than I thought it was. Yeah, there's several conferences listed here. I think a couple of these are not C++ specific. I think NDC Oslo is kind of just a general developers conference. But there were

several C++ conferences that I wasn't aware of. Mathieu, have you been to any of these European ones like Rob just mentioned that we haven't been to before, like NDC, ACCU? Not yet, but I'm doing ACCU this year for the first time. So ACCU? Not yet, but I'm doing ACCU this year for the first time.

So ACCU is not technically a C++ conference, but has lots of C++ content, is that correct? Yeah, that's what I saw from the program, yes. Okay. But that will be the first time for me, too. What will you be speaking on? I'm redoing an updated version of my talk on API and ABI. It's similar to the one I gave at CppCon and Meeting C++,

just updated with a few feedbacks I got, some things I left out. I got, I think, half an hour more for the comments, so obviously I will have to add a bit of content at the start. I'm planning on adding some stuff, notably about the inline namespace, which I completely forgot in my original talk, and maybe

delve a little more on the discussion towards the live at head stuff that's been presented at CppCon. Because, you know, my talk was originally a lot about SEMVer, and of course, my perspective changed after the keynote at CppCon this year.

Well, we'll definitely have to dig into that once we get into the interview. Yeah. And one more thing I wanted to mention here that Jens pointed out is C++ content seems to be on the rise. He's tracking the number of posts per day and how it went from five to almost six so far in 2018 and uh lots more video content as well as user groups are starting to record and release videos and obviously meaning c++ and pacific plus plus and cpp con are all posting their videos which just really creates a whole lot of content on the web. Jens also says here, while C++ has the best podcast with CppCast, it is sadly the only one. So if anyone is out there starting another C++ podcast, let us know. We'll give you a shout out.

Absolutely. Absolutely. Okay. And then the next thing I want to discuss is this project on GitHub called Inspector, which is a drop-anywhere C++ REPL. And we've talked about a couple different REPL projects before, but this one seems to be a little bit different. Yeah, I wish it had some sort of live video thing here to watch to see exactly how it plays out. But yeah, it looks like it's what it advertises itself to be. I didn't dig into the code at all,

but it allows you to place these include inspector lines, and those wind up being kind of like breakpoints in the REPL, I guess. And I'm not really sure how that works. It seems interesting, though. Do you have anything you want to talk about with this one, Mathieu? No, I just saw it recently. I didn't have the time to see

what it does. I see notice here that it requires libcling, which we have mentioned before. Okay, and then the last article is on CPP depend, and this is the 15 C++11 features you must really use in your C++ projects. And I guess this is aimed at developers who haven't upgraded or maybe just looking to upgrade from C++ 98 to 11 or 14.

And it's just highlighting a whole bunch of the kind of must-use features. It's a pretty brief article. I thought maybe it should have gone to a little more detail on to why you might want to use some of these features instead of just quickly explaining what they do. But I definitely agree with the list.

How about you, Jason? 99% of it. He mentioned shared pointer or sorry, the author mentioned shared pointer. I'm not sure who it is. And doesn't mention unique pointer. If you need to do heap allocation,

unique pointer should probably be your default. Right. Yeah, that caught my eye too when I read it. I found it strange that there is a shell pointer before anything else in terms of smart pointers. So I must admit that I feel like I live perhaps a little bit outside of the real world here because most of the work that I do these days

is research and somewhat theoretical for how to best practice the C++. What is, Mathieu, what is your experience? Are you fully post-C++14, C++17 world at this point? Well, that depends on where you call me. If I follow it or if I use it on my day-to-day work, which is not the same question, actually. What's it like in the real world at work, I guess, is the question.

In the real world, we just migrated to C++11 last month. Oh, wow. Yeah, yeah. We had Solaris to support for many many years but with all the layoffs at Solaris after I think was last year, some news that hit up about Oracle, clinging up a lot of the Solaris division. Finally, we managed to decide our clients to stop asking for Solaris and switch to Linux, which helped us

a lot because the C++ compiler on Solaris was very old and we had a hard time using C++11 with it. Okay. I haven't had to use Solaris since pre-C++11 with it. Okay. I haven't had to use Solaris since pre-C++11 world,

so that didn't ever affect me. But I have definitely gotten the impression it's a little sad because where I live is one of Sun's old headquarters here where a lot of Solaris development was done in the Denver area. And to see that Solaris is kind of a dead

system at this point, and hopefully there's no listeners that'll think that. But it does seem to be dying, really. Oracle is slowly killing it off, basically. Yeah, that's the impression I got. And even the compilers, we tried to use

the new version which provides C++11. And I think they were starting internally to use a lot of GCC's code instead of their own front end because they could not catch up fast enough. I guess maybe not enough people or investments on that area. I guess maybe C was more important

to continue building stuff like Oracle database, which is, I think, mainly C. Right. Well, and I think, mainly C. Right. Well, and I think GCC officially supports Solaris too also, so there could be a strong argument for just using GCC when you can. Yeah, that was our backup plan if Solaris would still be needed. I ran some benchmarks, and I actually had some interesting findings. For example, I found the Stepanov test.

I don't know if you heard about it. It's an old test that Alex Stepanov did. I think it was when the STL was released, basically, to help compiler developers make sure that they could nicely optimize templates and all that stuff. It's just basically a front-end test. Can you fold an algorithm call with iterators, that kind of thing. And the performance was so better on GCC compared to the Solaris one.

And I think one of the big reasons is that the front-end was poorly optimized for that. Even on Spark, which GCC is really not a big known for its big backend on Spark. It was faster with any of those on GCC just because the inlining was apparently much better. Right. That makes

sense. I mean, if you can eliminate so much code at the front, then it doesn't matter how well you optimize the assembly to some extent. Okay. Well, let's start talking a little bit more about some of the conference talks you've given recently, Mathieu. Yeah, you gave a really interesting talk

about API and ABI versioning, which is something that maybe comes up a lot in discussion, but maybe a lot of developers don't fully understand what the difference is. So do you want to maybe start off there? Yeah. Yeah, that's, well, basically my idea was to just go back to the basics of what's basically an API, what does it mean when you write a function, put it in some header files and

say, okay, this is public, you guys can use it, and especially what it means over time. Because in my career, that's something I thought maybe sometime was not so thought about. I come back from very early, I did open source stuff, which, of course, API was a huge concern. You cannot just break something and then put it on CVS or whatever it was at the time, because it would break for everybody. But when you're in a company which has maybe one big centralized code line and you can possibly patch all the colors at the time or something like that, you don't think as much about APIs.

You just think like, okay, does it compile? Yes. Did I patch the color? All right. And that's not the same mental model I think behind it and one of the the goal of my talks was to help people think about

what interface they do provide for people especially since C++ tends to go more and more header only so there is more and more code that you can actually look at directly and not just have the old-fashioned C separation

between really your declaration of your interface and its implementation, which would be in another file, something you may not even have the right to see because you get a binary or whatever. So, yeah, I tried to explain really the idea behind that. And of course, then I moved on to the ABI for binary compatibility because that's, I guess, also a big concern for a lot of people. We cannot just afford to tell people to recompile everything when something changes.

And that's not a position you want to be on, having to release a new binary and not having any idea if it you want to be on having to release a new binary and not having any idea if it's going to be compatible or not if you need to patch somebody real quick in production So these are really like two different things, the binary

compatibility versus the API Yes and no because basically a lot of the breakage of the API will come to change due to your API. Okay. For example, if you rename a function, well, that's maybe the dumbest example, but if you rename a function, the symbol will not appear in the binary, so people will not be able to run it.

That's the most trivial case. But then you can go into more subtle change. For example, if you replace, let's say, you had a function which accepted a short, and now you take an int. On the AP side, it's usually considered safe because you just accept a wider range of numbers. Nothing in the contract is violated.

But when you recompile that with the mangling stuff, you end up with a symbol that has a different name because of how the overloading works in C++. And suddenly it doesn't work, even if technically you would have been able to handle it on the source side. So are there situations where you could break ABI compatibility

in a way that's not detected at link time? If your client recompiles, you mean? Yeah. Well, no. The big problem with ABI is basically solved if you recompile everything. The ABI is basically solved if you recompile everything. The ABI is really a problem about you compile something against a version of a library

and later in the future you want to upgrade that library but you don't want to recompile everything that depends on it. That's a use case that comes a lot for low level libraries like for example take OpenSSL. It's very low level. Everything on your system is going example, take OpenSSL. It's very low-level. Everything on your system is

going to depend on OpenSSL almost at some point because it needs some cryptography. Almost everything today tries to do at least a bit of crypto. So you can't tell someone, well, there is a patch for OpenSSL, please recompile everything on your system that's your security. You can't do that. So you need to be able

to give him a new binary that will still work with everything he already has. And that's when you start thinking about, okay, is it ABI compatible or not? Okay. So you mentioned that with something that's in wide use as OpenSSL, that it's important to maintain this ABI compatibility. What is your experience with other libraries

that you might ship to your clients? Well, what can I say? We have a very big code line and we deliver a very big application with lots of modules and from time to time, yes, we have

clients, especially in the past, maybe not so much now, but for which when you said, well, let me send you back this 2 gigabyte new binary and just please install it in all your servers, that would have been a problem. So he would just say, no, no, can I just replace the one DLL or the one shared library that's affected by the problem and just that and not please reinstall everything.

So how often does that come up? I'm curious. Well, not so much lately. Most of the time, I think we don't have to do as much as hot fixing. But then again, I moved away from that because for the last two years, I'm away from the business and I'm more on the technical layers, which if you break something, you know it's

well before it goes into production because everybody's using it. So when I broke something on the database or on the HTTP API, something will flare much, much earlier in the process. And I never had cases when it went to production and I had to do a binary release without just

about delivering everything. Are there any tools you could use to check if your API has changed in the DLL or something like that? I think there is a tool. to check if your ABI has changed in the DLL or something like that? I think there is a tool. I think there is an automatic tool that should be able to do that for you because when I stumbled for my research for this article, I found something on the Internet that's used, I think, for JLipsey, for example, and I think also for the standard Lipsy++

to tell you the compatibility between the versions. And I wonder if it's not maybe a KDE project or something. I can't remember exactly, but I think there is a tool that's supposed to analyze both

and tell you if it's ABI compatible or not. It's not easy because just analyzing two binaries is not enough. You also have to look at the source of every structure that's exchanged through the ABI

and make sure that you did not reorder the members or change the definition size or something like that. So it's a tedious analysis, but I don't think theoretically something would stop you to, would stop a tool from being able to tell you that. I think there is one, but I never used it. Okay.

All right, so that's an interesting point, though. The mangled name of a function could stay exactly the same, but the meaning of all the parameters could be completely different. Yes, yes. Okay. Because the mangling only contains the type, not the names. So if you had, like, I don't know, probably not a good example, but if you had, like, three integers and just flipped them around,

the API is still the same, and it still works. In my talk, that's what I call one of the most evil things you can do when you change your API, is you do something that will pass through the compiler because you did not replace the type by an incompatible type or something like that. You do basically, you change your behavior, but there is nothing observable

from the perspective of somebody who just compiles all links. And then that means that if it doesn't have a test to catch that, you're out of luck. And what I recommend is to never do something like this. If you have to change your API, because sometimes you may want to change your API,

just break it. Just break it as hard as you can to make sure that it will raise a compiler error and the users will have to say, okay, why doesn't it compile? Wait, the signature is not what I remember. And then maybe we'll be able to look at your documentation and say, oh, yes, you changed that definition, and I have to adapt.

But by default, I don't think many people will go and read your change log or whatever because they're great. It's just, okay, does it compile? Yes. And if you pass through the test, it can end up in production. And if you just flip the meaning of a Boolean from true to false or something like that, the example I made in my talk is that you can end up selling the bank instead of buying it

or something like that, because you just change one meaning without changing any type. Yeah, so then your client might use the same library or use the new version of the library. Yeah. And from their perspective, the ABI is exactly the same. The API still even looks the same.

Yes, and now your function does not do the same thing. Possibly the arguments have lost their meaning or maybe the complexity has changed. A lot of things have changed, basically. Well, if you go back to the notion of contracts that are starting to appear, well, more than starting in some talks,

the idea is that there are, I think, two parts in a contract. There's the part that you can see, the part that your compiler will warn you about, like the name of the function is not the same, the type of the parameters is not the same, and there is all the things that you say you will do, but there is nothing at compile time

that checks that you actually do it. It could go even further, like, for example, you can just change what the function does, but not change the signature. Like, one of the stupid examples I gave usually is like, in the past, you did the sum, and now you do a max. You just, okay, that still compiles. That just does not return the same number.

But you have changed the API, but nothing on Earth will flare unless there is a test that actually tries that, and that's a dangerous thing. Usually, it's more subtle than just a dummy change, like a plus and a max, but that could happen. So perhaps maybe also insidious, if I'm understanding correctly, is you have, and you mentioned this briefly, you have a struct that you added a member to,

and you're passing that by value to some of the functions in your ABI. Then from the library perspective, the ABI hasn't changed, but the layout of that object has changed, and now what, you end up with stack corruption or something? Technically, first it could also happen if you go by pointer or by reference, because basically it's just two different binaries with two different understandings of what's the offset and what's the size of each member. So basically, you're asking for member A in the struct,

in the first library, it says, okay, it's four bytes at offset one. And the second one will say, no, no, there's a new member, it's offset two, and it's still four bytes. Basically, you disagree on the representation of the data. And you're just looking at bytes a different way and that cannot end well.

But the problem is it's... Once it's translated into assembly and to binary, your compiler can just interpret bytes the way it wants and the type system is out of the scope. So a lot of things can happen and maybe you would just be lucky and eat some zeros that

have been left out for padding, or maybe you will end up inverting two amounts in an order or something like that. Well, so then your advice to just, if you're going to break the API to break it hard, how do you deal with something like that, where you're just doing a restructuring of your state object okay um so uh for that I would say uh if you want the binary compatibility uh for people then you have to know uh when you break the, and then you have to go for versioning. Like say, okay, this is a new major release. This is not compatible.

Do not even try to link it with. And maybe you can have some sanity check or something at some point to just make sure that when your program starts, it asks dynamically the version of the loaded library to make sure that it's the one it expected or a compatible one. Or just maybe put a symbol with a certain value that will not be there if you change the major. Just something that makes sure that the program won't start. I wanted to interrupt this discussion for just a moment to bring you a word from our sponsors. Jason and I had a super interesting podcast with Arno back in January.

He's the CTO of ThinkCell, the number one graphic software used in consulting firms. He's still looking to grow his team, and what they offer is quite appealing. They work on extremely challenging C++ development tasks, and they offer more than pleasant working conditions. With them, you could be working on things like solving linear constraints for automated layout or reverse engineering PowerPoint with IDA. They have a large library of generic algorithms extending the functionality of STL and Boost, in particular improving Boost ranges. So what you contribute could be used immediately throughout their half million lines of product code. They're always using the latest C++ features that are available in the latest version of Visual C++

and Clang. That's because ThinkCell software is cross-platform for windows and mac as for working conditions i'm talking about no overtime no meetings no deadlines and any technical issues can be discussed directly with arno with them you join a team of 20 plus knowledgeable and creative programmers with a mix of young and experienced developers coming from 20 different nationalities from all over the world it's a family-friendly environment and they pay 120k euro annually after just one year. So if you like what you've heard so far, go to thinkcell.com slash cppcast to find out all about the job and apply today. So you were talking a little bit about, you know, how you should make a change if you're going to break the API, break it hard, and how users are probably going to be

looking at your documentation if you've introduced some type of breaking change to your API. And it just kind of made me think about Titus Winter's talk, which I believe was before yours at CppCon 2017, and how his recommendation is to kind of just, if you're going to break your API, you should also be distributing a script to fix API usage. And I was wondering what your thoughts are on some of his takes on Sember and everything.

That talk was a very interesting experience because I remember I was in the audience. I think that was my first talk at CppCon or something like that. And I was in the audience. The next day, I was scheduled to talk about Sember and ABI versioning. And Ted Titus comes up on the stage and starts saying

things like okay this is how you do it, this is my keynote and no you don't do ABI compatibility and no you definitely not use oh my god what am I going to talk tomorrow about but as it turns out there is middle ground to be found and I think his stance on

automatic patching is very interesting obviously Google has the infrastructure to run automatically lots of commits run by bots. I think I read somewhere that they have today more bots doing changes than humans in their conlines every day. Wow. So, yes. than humans in their con lines every day. Wow. So, yes. But, yes, that's a very interesting take, actually. You start to see your code almost like a database.

You know, like when you change a database, you're supposed to provide some kind of script to do a migration. But basically, that's the same idea. You say, okay, you want to take my new version of my library? Please run this migration script on your code. You start seeing code as say, okay, you want to take my new version of my library, please run this migration script on your code. You start seeing code as data basically, and how to migrate this data from one model to the new one to still

be compatible with whatever you're going to do next. I think that's a very interesting take. It's maybe today harder to write a code migration script than a SQL migration script. But I don't know.

Maybe in the future things will be easier to do. I think that's an interesting take. Then again, he also advocates always recompile, which I guess from the feedbacks I got from my talks, not everybody agrees on that, at least for financial reasons. Right. I think the point Titus is trying to make

is that no matter how much your server costs, it will still cost you less than a fully paid developer working on those kinds of bugs and problems and having to think about that. I did not run the numbers. I'm just going to have to stress his experience on that. And I'm not sure this case has been argued everywhere,

and there are certainly a lot of companies who do not think that way and still believe that every binary stuff you can just take and reuse is money saved. Well, and I think your point about OpenSSL or some very widely used thing where being able to quickly and easily distribute a security patch is important.

There are clearly parts of the world where that matters. Yeah, yeah, clearly. On the phone, for example, on your Android phone, maybe you don't want to reinstall everything. I mean, I got fiber optics on my phone, for example, on your Android phone, maybe you don't want to reinstall everything. I mean, I got fiber optics on my home like maybe one year ago, and we got 4G, what, three, two years ago.

I can't even think about like re-downloading Windows every two days to every application built on top of it because somebody patched a kernel. That's just not possible. Even with fiber optics of whatever connection you get.

Even talking about parts of the world where DSL is not available everywhere. There's actually one of my Android devices I ended up getting rid of because no one released a Heartbleed patch for it. So it specifically hit me in that kind of way.

Yeah, definitely. And you had another talk, which was about Modern CMake. Do you want to tell us a little bit about that one? Yeah, Modern CMake, yes. Basically, it's not something I invented, or at least I did not invent the term it's something I found during my research on CMake

I think it's from someone Steve Riviere or something I'm not sure of his name he's actually scheduled to speak at ACCU about CMake again too bad I think he's just at the same time as me,

so I won't be able to attend and vice versa. But he's the first one, I think, who introduced the concept. And from what I remember, I think he even made the patch inside CMake to make it work. I don't think it came from the actual maintainers. It was more...

Well, the idea is that CMake, the traditional CMake is, well, most like Make, it's just script. You just run this command, then run this command, then use those flags and compile with these flags. And if you look at build system now, the new ones that come up, the one that Google released, and the things that resemble it a lot

that came up from, I think, Facebook and maybe, was it Twitter? I can't remember. Like Pence and Buck, things like that. If I try to think about, more about, again, as a schema, as architecture, architecture basically like what what is your code architecture what's a module what's inside it just tell me what what are your modules tell

me what they contain and i will handle the technical bits of when to call cc with which flags and coming from a big code line where we did not have that, but more or less just ran build scripts, well, you can end up with lots of ugly stuff because adding a new dependency is not even something you have to do consciously. You know what I mean? You can just pound include something, the auto-completion in your ID

will fire up for almost everything on the code line. So as soon as you can find it on Visual Studio or whatever you use, you just use it, and sometimes without second thoughts. And you do that for, I don't know, any number of years, and you end up with circular dependencies or spaghetti code all over the place. And the point of my talk with 2C make practice is really take back control of your dependency graph. When you think about something, think about

modules. Don't say I need this minus I flag, minus L flag, whatever to build. Tell him this is my module, this is my sources, this is my public headers, this is my private headers, this is the name of the modules I need as dependencies.

And then let him resolve, translate that into a graph, see if there is any inconsistencies, see if there are any cycles, and then if everything is fine, just build. It's something a robot can do much better than you or me. So what should developers be doing to take advantage of some of these modern CMake best practices? Well, it's simple. Basically, I try to give a lineup in my talk.

The idea is that simply there are a few keywords in CMake that have been done specifically for that. Which say, I think it's targets in link libraries. It basically says, this is my library, and this is the library it depends on. And that's almost the only thing you should declare once you declare a project. Basically, you say, okay, I add my project, which is usually a library. Sometimes it's an executable, but if you think about big projects, it's a lot of libraries and sometimes an executable at the end. So you just say, okay, add library.

This is my sources. Then you may have this macro which is called target include directories, which says this is the public includes I want people to be able to see if they use my module themselves. And then you finally say target link libraries, which is take those libraries consume those libraries as a client and that's almost all you need

to do you try to go as declarative as you can instead of thinking of CMaker as a scripting language because for once it's not a good scripting language and two it's much easier to think

and to reason about something that's descriptive rather than code. Because, well, code is code, it's showing complete, and trying to run an analysis on that and see if the thing is actually sane is much harder. CMake is in this unfortunate place where Visual Studio is also at the moment, where anything you try to Google for is finding out-of-date documentation. Yes.

Finding the most up-to-date stuff is very difficult. And since Visual Studio rearranged their documentation layout on their website, they have the same problem. But do you have any recommendations for resources where people can go to? This is a canonical example of good, modern CMake best practices. Obviously, there are a few talks.

The one I gave, the one that Daniel Pfeiffer gave I think at CPP Now. I think I was in that one. Yeah, I think it was one of the most helpful at the conference which was both surprising and

not surprising because you know at CPP now you expect like the best talk to be something about very advanced templates or whatever and it was a talk about build systems and I think that made me think about how much people need to need help on this and need good tutorials

that was the first motivation of my talk, actually. After how many times, how long I spent trying to learn that stuff, especially with the state of documentation, I said, I cannot just let that go for me. I need to capitalize on that.

The documentation is actually only helpful if you know... It's helpful as a reference. It's not very good as a tutorial. Right. And of course, there are Steve's posts about Modern CMake. I can send you the link after the cast if you want to add up to the end.

That's basically the article that inspired me to start all this. Okay. Are there any tools that you know of that could help you? Like, I would love to see Cmake modernize like Cling modernize. That would just be... I don't know about any tool. I met someone in Stockholm who started working on something to more or less

translate CMake from a scripting language into a declarative language. Basically, it created a bunch of functions and macros where you just have to declare a project, and it translates that into the CMake script, but all you see from your perspective is just declarations,

which usually makes the things much easier to understand. I guess I'm going to have to follow up on that when I'm there to see if that can maybe be improved and generalized.

That ends up getting us into the situation of almost like a meta build system or something like that yes I think we kind of agree that CMake

is not really the build system we want but it's the one we have to use for no reason at least before for the reason that everybody else, or, I mean, a lot of other projects are using it, and it's just more easier to integrate.

All right. We talked last week about how the modules TS was getting updated, and it seems to be progressing along on C++20 standardization. As someone who thinks a lot about build systems and package management, I'm wondering what your take is on the modules TS. I did not read the last version, but I remember having a discussion about the TS with Isabella Muerte at CppCon just after a talk.

I think she mentioned that when she came by for your cast. And then again, I'm just going to end up paraphrasing what she says but uh i think it seemed like it's not trying to solve that issue uh it's more seems to be trying to to go for maybe faster compile times uh getting rid of macros, and possibly standardizing the old PCH that appears a lot on Windows but are completely absent on other systems. I don't think build system and package management is something that the committee wants to look at. Do you think it's something they should look at?

I'm pretty sure it does, yes. I think, yeah, well, I don't want to sound fatalist, but I'm afraid that if something is going to end up being like the big pain and the big letdown in C++, it's not because concepts are late or because we don't get modules. I think it's going to be because it's just way too hard for somebody to reuse the code of others.

I've seen a lot of people, when they introduce their libraries at conferences, the two main arguments they always say is, first, it's header-only, and two, it has no dependencies. And every time I hear it i'm like what uh either only okay maybe it's something people like to do today i'm not sure it's for the good for the for the good reasons but why not but no dependencies just strike me as odd i mean i've been told from a very young age that it's better to to do small stuff that does something great and then build on top of the rest

for your project. And what I see is a lot of people saying, nope, I prefer to rewrite it than to have to go through the pain of actually reusing something. Like the cost of reuse is basically bigger than the cost of rewrite. And that's

I think that's a big problem. I guess to to be fair, though, if you say, in current C++, if you say no external dependencies, you can still mean all of C++ 17, standard file system, thread, the regular

expression, standard library, and all that, yeah. Yeah, that helps, and I guess with 20, that will help even better, because we will actually have network. Because today you cannot do network with standard C++. Right. And I don't know a lot of applications that don't have to interact with the rest of the world.

Yeah. Yeah. That'd be interesting to see. Hopefully we do get ASIO and C++20. I don't think it's been officially approved yet for 20. I don't think so. I don't think it's been officially approved yet for 20. I don't think so. I don't think so, no, but I think Boost modified

its code to follow up in the last version to align ASU on the TS, I think. They renamed some variables and some structs to that end. So I guess hopefully we are moving toward it.

Okay. So you also gave a lightning talk at meeting C plus plus on the need for a Patrick package manager interface. Yes. What was that about? Well, I think I started working a bit with the Conan guys on trying to stand and the bean craft bean crafter the Conan guys on trying to stand the Beancrafters initiative

of people trying to... Basically, it's a group of open-source developers who are trying to push good recipes of Conan packages on the Conan centers to help the dev who don't really have the time to do that. And one of the issues I realized is that for every package I have to write,

I have to script. There is no standard about... Even if they all use CMake, at some point, I will need to script around something. And so my idea is that if we could standardize

the way you invoke a build system, like this is the tool chain, those are the dependencies you have, this is the options the user wants, can you work with it or not? If not, can you tell me what you need and I may try to install it for you? And once that is done and built,

this is the install there, tell me where it is done and built, this is the installed here. Tell me where it is installed and give me something like a PKG config or a descriptor about the ABI flags that should be passed to the users. And once we have some kind of interface, it doesn't

matter which kind of build system you use, as long as it follows the convention, it could be packaged almost automatically. And even better than that I guess it would help us more easily to change our main build system.

For example if we start saying no CMake really we can't use CMake we are fed up with it. I'm saying CMake as an example because that's the biggest one I see but that could be anyone. We could just say well it's I'm saying CMake as an example because that's the biggest one I see, but that could be anyone. We could just say, well, it's the good old interface versus implementation principle. We could just say, all right,

it's not because I changed the build system of my package today that everything will break for my clients. Whereas today, if I replace CMake for my project, like Boost is going to do, for example, replacing Boost build with CMake, that's going to have a huge impact on a lot of builds, I guess, because they will have to adapt. If we had an

interface, they just would not have to do anything. They could just take a new release and say, okay, done. So do you think we'll actually get any movement on something like that? I don't know. I started my talk and my post on this to see, I guess this to test the waters and see how people will react to this.

I got a few encouragements and also some Linus Torvald codes like talk is cheap, show me the code, something like that. So I guess if I could find time to write a prototype, maybe for CMake or for something like that, that could help. I wish I had some actual answers from people who maintain build systems, but so far I haven't got any. Yeah, I'm only aware of a few people who actively maintain build systems

who also come to conferences and that kind of thing. Yeah. Yeah. Interesting. That's why I said that maybe the committee could help in that way of trying to push a standard. I know it's a very difficult problem because it comes up with a lot of how do we offer a migration path for legacy code. And that's something like, for example, you say Rust or Ruby didn't have a lot to think about because they just had it from day one. When you're not coming with 30 years' baggage

of already existing libraries, it's easy to say this is the standard and everybody's going to use it. There won't be any discussion. Today, if you come and say this is the standard and everybody's going to use it. There won't be any discussion. Today, if you come and say, this is the standard for your build systems, I guess like 99% of the world will say, no, because my build doesn't work that way. Right. And this is where

I try to find maybe some middle ground and just enough concession for the thing to be not too painful or maybe not painful at all if possible, but still get the ball rolling. Right. That sounds like a good idea to me.

It would be nice to see some movement, but I guess someone really needs to kind of carry the torch if they want to try to bring something like that to standardization. And convince millions of people to agree with them. Yes. Yeah, I guess I'm probably just going to end up with the old XKCD about standards, you know, like the one that comes up every two talks. I just put it in mind just as a preemptive measure. I'd say, okay, I know you're going to say that, so just let's get on with it.

Right. Okay, well, it's been great talking to you today, Mathieu. Obviously, we'll put links to some of your talks in the show notes and some of the blog posts you made. Is there anything else you want to discuss before we let you go? No, specifically, no. Okay, well, it's been great having you on. Yeah, thanks for having us. Thanks so much for listening in as we chat about C++.

I'd love to hear what you think of the podcast. Please let me know if we're discussing the stuff you're interested in. or if you have a suggestion for a topic, I'd love to hear about that too. You can email all your thoughts to feedback at cppcast.com. I'd also appreciate if you like CppCast on Facebook and follow CppCast on Twitter. You can also follow me at Rob W. Irving and Jason at Leftkiss on Twitter. And And of course you can find all that info and the show notes on the podcast website at cppcast.com. Theme music for this episode is provided by podcastthemes.com.