CppCast - Conan 2.0

Episode Date: May 12, 2023

Luis Campos joins Phil and Timur. After some compiler and language news, we chat with Luis about the recent Conan 2.0 release, the state of package management in C++, today and what's happening in the world of C++ standards that may impact it in the future. News GCC 13 released CppFront "spring update" Mojo - a new language with Python-like syntax but runs like C++ CrabLang - a fork of Rust Timur's Undefined Behaviour survey Links Conan 2.0 P2656R2 - "C++ Ecosystem International Standard" Conan on Twitter "A Lock-free Atomic shared_ptr" - Timur's C++ Now 2022 talk "Advanced Dependencies Model in Conan 2.0" - Diego's ACCU 2022 talk

episode 360 of cpp cast with guest luis caro campos recorded 4th of may 2023 this episode is sponsored by jabrains smart ide's to help with c++ and sonar the home of clean code In this episode, we talk about new C++-related compiler releases and some new programming languages. Then we are joined by Luis Caro Campos. Luis talks to us about the new version 2.0 of Conan, the C and C++ package manager. Welcome to episode 360 of CppCast, the first podcast for C++ developers by C++ developers. I'm your host, Timo Dummler, joined by my co-host, Phil Nash. Phil, how are you doing today?

I'm all right, Timo. I've been wrapping stuff up before my next trip. I'm about to start what I've been calling my Scandinavian tour next week. Although, because I'm going to be away next week when we would normally be recording, we're recording the week before. So by the time you hear this, I'll have already done it. So I'm not quite sure how that works. But if I've already seen any of you in Copenhagen or Malmö, then thanks for coming. So how's things up in your far corner of the Nordic countries, Timo? So I'm good. I have something very similar similar going on i was traveling a lot last week um i was actually in madrid at using stud cpp

that's the conference organized by the carlos the third university of madrid and that was really good and literally just now i came back from stockholm where i was giving a talk um last night about c++ and safety which is uh was kind of a draft slash rehearsal version of the talk I'm going to give at your conference, Phil. CPP on C. Iron out all the bugs. And then just in two days, I'm going to be flying to Aspen, Colorado for C++ Now.

So when this episode comes out next week, I will be there. Yeah, sounds very busy for you as well. Yeah. Yeah, I'm looking forward to some vacation. It's been a lot of business travel lately. this episode comes out next week i will be there yeah sounds very busy for you as well yeah yeah i'm looking forward to some uh vacation it's been a lot of business travel lately right right so at the top of every episode i'd like to read a piece of feedback and this one is from an email that we received actually quite a while ago back in january um as a response to the episode where phil and i took over the show. And Robert gave us the following feedback. Robert said that I would like to see more attention paid to IDEs, tools, and all their aspects.

Well, thank you, Robert, for the feedback. We actually fully agree with you. We like to talk about tools. It so happens that both Phil and I actually work in the tooling space as well. So that's why we're actually launching, starting today, a new series of episodes about tooling. So we're going to be talking about package managers, IDEs, build systems, and things like that over the next few months.

I think the idea roughly is to have every other episode be about that and then every episode in between to be about something else so that it's not just about tooling obviously there's lots of other interesting topics so it's a little bit similar like the mini series we did on the so-called successor languages for c++ yeah so this one is going to be about tooling so i hope you're going to enjoy that looking forward to it we'd like to hear your thoughts about the show you can always reach out to us on Twitter or Mastodon or email us at feedback at tppcast.com. And joining us today is Luis. Luis is an electronics and computer engineer currently working as software tech lead in the Conan team at JFrog. He started his journey with

C++ in the university doing research in the field of computer vision. Since 2015, he has been based in the UK and has worked in the field of 3D scanning and more recently in the field of robotics, autonomous driving at Oxbotica before joining the Conan team at JFRO in 2022. Over the years, Luis has become concerned with the problem of enabling C++ software development at scale, working on the tooling and processes that enable large teams of C++ developers to focus on writing code. Luis, welcome to the show. Hi, thank you very much for having me. So obviously we're going to be mostly talking about your work at Conan and JFrog today,

but I'm also interested in the autonomous driving you mentioned. What was your involvement there? Well, actually, when I joined, and I think it was the week that I joined, the team was deciding on what to do with C++ dependency manager so one of my colleagues had quite a really good review of the tools available at the time and that day I was like how about Conan how about we integrate Conan because how I used it before in my previous job so my involvement grew from that into sort of uh looking into the software infrastructure side of things which is what we call the team uh that handled anything to do with

the build system and with dependencies and ci and all that stuff yeah or devops we often call it these days i know yeah we've somehow we resisted to call it devops for some reason well done okay so louis we'll get more into your bio and your work at JFrog and Conan in just a few minutes, but we do have a couple of news articles to talk about. So feel free to comment on any of these, okay?

Perfect. All right. So the first news item for today is that GCC 13 got released. So we had the release of Clang and LLVM 16 not too long ago, and now GCC13 came out. They added lots of C++23

language features. Among others, they added AttributeAssume, which I'm very happy about. This is a feature that is standardized, so nice that we have a first compiler actually supporting that. They also added StaticOperator paren paren and StaticOperator square bracket. Lots of all the other kind of smaller C++23 features.

There's a long list on the kind of change, kind of release notes document. And actually another one of those that I found particularly interesting, because I only found out about it very, very recently, is they implemented this C++23 feature called relaxinging ConstExpert Restrictions. It's pay-by-peer 2448. And basically what this does is it allows you now to declare a function as constexpert

even if it does non-constexpert things like print hello world to see out or something like that, which is obviously not you can't do that at compile time. But you can still now, since C++23, declare that function as constexpr and as long as you don't try to actually call it at compile time and you only call it at runtime that's now fine and they implemented that and according to the gcc bug tracker it was actually wasn't trivial to implement but it gets

rid of kind of annoying uh errors in certain situations and it's kind of a cool cleanup, in my opinion. Yeah, that one flew under my radar as well. But I presume it's similar to what we got in C++20 where you could throw exceptions as long as you didn't actually throw the exception. Yeah. Is that similar? Will it just not compile then if you do end up calling the runtime functions?

Yeah, so I encountered this actually recently when we were discussing contracts and topic came up of like oh how do you uh because we're still designing the feature right we are aiming for c++ 26 and the question came up so how do we uh evaluate contracts at compile time and then we had a discussion about this and you know somebody said uh oh yeah but then then in a constexpr context, you can do this. And I was like, hang on a minute, like what? And then I started digging and I found that, oh yeah, it's a thing now since C++23 and GCC now actually implemented this. So that was interesting. So they also added a bunch of C++ 23 library features, like monadic operations for expected,

like and then or else, like those things, which we got into the C++ 23 standard at the very last minute. Yeah. Constexpr bit set, a lot of new ranges stuff. I'm not going to list that all, but it's a lot. And also some C++ 20 features that were actually not in GCC yet, notably std format, which is now in GCC, which is really cool.

Yay. And they also have some other improvements and additions unrelated to the new standard. They improved the static analysis, they improved

OpenMP support, they improved support for other languages like Ada, Fortran, etc. Big release. Quite a major release. That's really cool.

I think it's going to take about a nanosecond until we get a request to start building packages in Content Center with GCC 13, so I'm looking forward to that one. Good luck. Good luck with that. Thanks. And there is actually another news item

that has to do with compilers. Actually, Herb Sutter, whom we had on the show, I think, was it four weeks ago, six weeks ago? I lose tracks. Very recently, who was talking to us about his CPP2 language and CPP Front compiler, has just released a CPP Front Spring update with what's been happening with CPP2 and CPP Front.

There's lots of exciting new features there. So CPP2 can now do user-defined types, chain comparisons, lots of other stuff. He has an initial version now of a reflection API in there, which is something that we are very stuck on in C++ 1 at the moment, but C++ 2 is leaping ahead. He actually has already 10 working compile-time meta functions,

which are all coming from P0707, which was his metaclasses proposal. So if people are asking, like, what's happening with his metaclasses proposal? Well, those features are all now going into CPP2, I guess is the answer. And also his list of collaborators for that project is actually growing really steadily,

even though he keeps saying that it's just this private experiment and all that. But there's definitely more people involved now, which is also kind of cool. Don't all things start as a private experiment? And then before you know it, everybody's using it? Only the successful things. Exactly. Yeah, I thought it was amusing that in the same release, we get classes, basically. We also get reflection and meta classes, which seems like quite a big jump. But I suppose that is the point release we get classes basically what we also get reflection and meta classes which seems like quite a big jump but i suppose that is the point of cpp2 speaking of

programming languages and new programming languages this is not really c++ news but it's kind of vaguely related so i just wanted to bring it up so there's actually a new programming language which which was announced for all of ai i guess that's kind of the tagline. It's called Mojo. It's available at modular.com slash Mojo. And it's similar to Python, but fast. That's basically the premise. And it's not just fast. So it's not just focused on performance in general,

but it's actually aimed at heterogeneous programming. So it's meant to run on all kinds of different hardware and utilizing it properly, like CPUs, multicore, but also GPUs, kind of exotic accelerator hardware and stuff like that. And so the aim is to achieve performance on par with C++ and CUDA, but without the complexity

and with a front-end that's kind of very similar to Python. So that's really exciting, I think. Absolutely. Those are some big promises when it comes to performance yeah yeah looking forward to yeah i think the last language that promised something vaguely similar was julia which i don't think did very well in the long run so hopefully they will have more success yeah you do see similar promises from time to time and usually it doesn't quite follow through but always exciting though so interesting to see where it goes yeah yeah so what it is basically from what i understand it's like a python like front end on top of llvm mler which is like the multi-level uh er uh for kind of running on different hardware and stuff. So yeah, let's see.

And another kind of news item that I just wanted to mention relating to another programming language is that it looks like Rust got forked. And I wasn't sure if that was maybe a joke. Maybe it is a joke. As you know, Phil, I'm not really good at getting jokes, but... Well, there's a joke in there about having a rusty fork. So apparently the rust community has quite a lot of uh illegal drama going on at the moment and that's kind of a result of that so the new rust fork is called crab lang or crab the crab language um and it's available at crab lang.org there's a a GitHub page. There's a long Reddit thread where this is being discussed.

I'm just going to cite from crablang.org. They say that our goal is not to fracture the community, but rather to provide an alternative for those who share our concerns and wish to have more freedom in using, creating with, and promoting the language without worrying about the litigation associated with trademark infringement. So, yeah, sounds like some legal drama indeed. So from what I gather, Brass has many, many good things,

but what people really, really like is the logo. That's interesting. It looks like this gives them a chance to keep the same style of logo. Absolutely. Right. keep the same style of logo absolutely right so i guess you will be um observing this drama from safe distance and wish them all the best um right and and then uh one thing that i just wanted to bring up quickly as i mentioned a couple episodes ago is that my um undefined behavior survey is actually still running it's a little survey it's just three questions uh one of behavior survey is actually still running. It's a little survey.

It's just three questions. One of them is actually optional. So it's really just two that you have to answer. It takes about two minutes. And it's a little survey about undefined behavior, basically what it means for you. And it's a little experiment I'm doing for my talk about C++ and safety,

which I'm going to be giving at CPP on C next month. So if you do want to help me out and have a more representative data set for this, then it would be very helpful if you could help me and fill out the survey. It only takes a couple of minutes, timo.audio.com. And I will reveal the results at CPP on C.

And I'll put that link in the show notes again. All right. And presumably if the survey never stops running, feel the results at cpp on c and i'll put that link in the show notes again all right and presumably if the survey never stops running that itself is undefined behavior isn't it well but not yeah but like an infinite loop is undefined behavior so then we can assume that the survey never happened and optimize it out of existence and that would be very unfortunate so i think i'd rather terminate it and write some output to okay um yeah so don't rely on it still running yeah right so um yeah that's all the news items that we have for this week so i guess we can transition to our main uh topic for today which is uh louis and his work at conan hello again louis hi how's it going good good so uh before

we get to um ask you stuff um i just wanted to share a story if i may about conan so um i haven't actually used conan very extensively but it literally saved me the one time where i really needed it and so you know i was kind of ignoring this whole stuff for a while. It's like, oh yeah, package manager, some weird like frog people. I mean, I don't really need that. I just write my own make files and I just have my own libraries and I know how to compile them, right? So that was going on for...

I wrote a song about that. Yes, you do. What could go wrong? So I was doing that for a while. And then I was preparing this talk about atomic shared pointer, lock-free atomic shared pointer, and how to implement it. And I was doing that for C++ Now. By the way, there is a C++ version, which is 60 minutes from that talk.

Don't watch that one. Watch the full 90-minute C++ Now version, because that's much more detailed. So I was preparing that talk. And what I was trying to do is I was trying to benchmark different implementations of Logflow Atomic Share Pointer. So I found three implementations, open source. Two of them were basically just, here's a header file.

They weren't really production ready. And I found only one implementation that was actually being used in production by Folly, the Facebook library. So I was like, okay, awesome. I need to like benchmark this, compare this with the other two implementations, compare this with my own implementation. It's a header only class, right? It's like atomic chat pointers,

like a template, fantastic. So I'm just going to take this class and compile it and benchmark it. And then it turned out that it was just a world of pain. Like I couldn't figure out because I was just doing CMake. I couldn't figure out how to integrate Folly into any of my stuff. I spent like a day on this. It just wouldn't work.

I asked my friend, Victor, who works at Facebook, hey, how do I use this? And he was like, I don't know. We have our own internal build system at Facebook. I don't know how you do this with CMake. We don't do this over here. I didn't find anything on the internet about this, apart from a few people who were apparently struggling as badly as I was. And then I was like, okay, screw this.

It's just a header. So I don't need to compile the actual library. Let me just pull out the header. Then the header was pulling in other headers. And then those headers were dependent on some macros, which were defined in yet other headers and some of these macros were apparently defined by the build system, which obviously I wasn't using

so nothing would compile. And I spent another day on this and it was just really very painful and I couldn't get anything to run and I had the talk coming up the next day. So I was like, what do I do? And then somebody said, why don't you try Conan? And so I read like a quick tutorial,

like how to get started. I wrote like this Conan file.txt, which just says, you know, I need like Folly and I'm using CMake. And then I did like pip install Conan. And then 10 minutes later it was working and I had like the library and it was actually compiling and running.

And I was kind of blown away by that and so ever since then uh I'm kind of really curious about what you guys are doing and uh I think you're you're awesome so definitely wanted to get get you on the show so I'm very happy that that you're here and yeah so you just had a major release didn't you absolutely yeah so like a couple months ago i think it was in mid-february we released uh conan 2.0 so full um of new features and excitement yeah and so i should also say is that i was talking to diego initially i've known diego for a while now and i was like oh diego yeah of course you could come on the show but you have kind of already been on the show we wanted to uh you know

have more people new people a chance to to speak so whom would you recommend and it would be ideal to have somebody who can really talk about conan 2.0 and everything that's going on there and then he said you need to talk to louis so it's because he knows i never shut up all right so for those of our listeners who are maybe not familiar with Conan, can you give us just a quick overview over what it is and what it does? Absolutely. Yeah. So like, just as you were telling in your story, right?

You started with, I was managing my own make files. What could go wrong? Turns out a lot of things can go wrong. And that story that you just told, you'll see variations of the same story in a lot of teams and a lot of companies. And the more dependencies you use or the more variables that you have, like maybe you're targeting multiple platforms. Maybe it's only Linux. Maybe you're mixing Windows again. That tends to become more complicated, if not unmanageable, depending on what you're doing. So Conan is a tool that allows you to abstract those things away from your build system right

anything related to building other people's code it will help you abstract away from your own build system and generate something that your own build system can consume so like it provides an abstraction that matches your own build system as well without having to worry to worry about the others but depending on the on which thing you ask me, I'll give you a different answer, but I think that's a good one based on the story you told earlier. Yeah, that's a good description.

So I remember when I was dealing with this horrible pain of trying to get Folly to compile, there was one problem where Folly would just pull in this other dependency. I don't remember which library it was, which was totally not needed for this one class template that I was trying to compile, but it would just pull that in. And, but then like my project was using another version of the same library and then they were like clashing and I was getting super weird linker errors. And I spent literally a day trying to

figure out what's, what's going on there. So how do you, how do you like resolve problems like that? How does that work? That, That particular problem is going to happen to basically any package maintainer, right? Well, especially we see that problem in Conan Center as well, where we actually make available both the Conan recipes and the packages for libraries, is we have to make sure that the build system or the libraries that we're building are not pulling things in.

So we have to find if it's happening, where it's's happening and disable that and make sure that they can find the version that we have elsewhere uh in current center precisely to avoid uh this sort of uh linker linker errors as well so well the way we solve that problem is by not like doing what you described i suppose and basically make sure that every dependency that you have is actually modeled correctly as a dependency graph wait so so do you actually go ahead take like each you know reasonably popular c++ library that's out there and just figure out how is it going to be build it how do you build it like how is it supposed to be built and how do we like tweak it such that it fits or absolutely do that work yourself um well we do have like a really uh active and engaged community in conan

center uh to do this as well but yeah absolutely and i'm pretty sure uh if anybody out there is listening that works let's say on the devian package maintenance or uh homebrew vc package conda, that particular problem is the same. If you really want to model the dependency, you're going to have to actually go and figure out whether this is happening or not. Fortunately for us, a lot of projects are

using more modern build systems, either modern CMake, as people call it, that may be using Mason, and they're following better practices, would say and not sort of bundling bundle those dependencies i sometimes feel like when a library author bundles of dependencies in the build system is because they've done what you described earlier i have my own make files uh right so they're thinking about their workflow of i'm developing

this library i'm i'm the end user but the reality is they're not the their workflow of, I'm developing this library, I'm the end user. But the reality is they're not the end user. The users of your library are the end users. And they may or may not be using the same dependency in a version that may or may not be compatible with the one you're using as well. So that's one of the problems we try to solve. Or at least Conan gives you the tools to solve that problem. And then the responsibilities are shifted

to the package maintenance, so to speak. So we already mentioned that Conan 2 is out, but what's actually new to make this a major release? Oh, there's a lot of things that are new in Conan 2. One of them is the tool itself.

I was just checking earlier. We've pretty much rewritten 80% of it, I think, if you measure by lines of code, which is arguably not the best way to measure these things. Over the years, and we have to keep in mind that Conan, I think the first

preview release came in mid-2016, so it's been over seven years now that Conan's been out there. And we very much tried to keep it sort of compatible, like Conan 1.x, keep it compatible, do not bring compatibility in any way with the way users were using it.

But over time, it became more difficult to release bug fixes, add new features, just because, you know, software over time, these things are difficult. So it was re-architected, especially the core components, which makes it a lot more flexible, both for us as maintainers of the tool, but also for the users as well. So one of the new features is extensibility

in ways that are easier to manage. Have your own custom CLI commands, for instance. So that's always been a really popular request. But also custom ways of modeling whether binaries are compatible with other variants of the same library, for instance. So maybe you can say, oh, a binary build with C++17 is compatible with a binary build with C++14.

Or any rules that you may have arbitrarily have in your own project, in your own team, you can actually express that in code away from the recipe, which was possible in Conan 1. And try to keep the logic of the recipes as clean as possible. It's ironic, actually, that it took a breaking change to be able to achieve greater compatibility. Absolutely, yeah. And this is not something that we take lightly.

As I mentioned before, keeping compatibility was always a goal. So bumping the major version to version 2 indeed caused some breaking changes. Some features have gone away. Some features that were experimental in Conan 1, like package and recipe revisions, are now fully enabled by default. We've been taking care, as I mentioned before,

of not having breaking changes. But with the new version of conan we have made some changes that obviously break how users use it we have at the very least provided a compatible compatible features in conan one so there's a subset of conan one that is very much one one compatible with conan two and that's the approach we have taken for instance in the recipes in conan center so at the moment, they are compatible with both versions. But also some features have gone away. Some features that used to be experimental are now actually the default in Conan 2

now that we've had a chance and users themselves have had a chance to provide feedback over the years as well. So is the typical user likely to need to make changes to migrate? Or is it just specific users who are going to be broken um typically users yes especially because one of the one of the biggest things that we've changed are the cmake integrations so they're likely to have to make changes to the recipes what we've seen is that they're not very invasive changes we do have in the documentation we have a guide, a migration guide to sort of some of the changes that you have to make. So what we hope is that the vast majority of users

don't have to make invasive changes to the recipes or their workflows, but they are going to have to make at least some changes indeed. And I believe there's mention of a new graph model. Can you say anything about that? Yes. And there's a very good talk about this that Diego did last year in ACCU. So I could just give a brief summary about this. So we've seen over the years, users have been using KanoNet scale.

So we have teams that maybe have five to 10 dependencies. We have teams that have like maybe 90, a hundred, something like that. And then we have teams that use Conan for both external libraries, like open source libraries and their own libraries and workflows. So one of the things that becomes challenging with this model is there's two aspects of this. It's like obviously how to propagate the flags down to the compiler and the linker where they are needed, but also the logic of what do you rebuild when something changes. When you have a complicated dependency graph

and you have it at that scale, there comes a point where you cannot afford to rebuild everything. And that's where the new graph model comes in, is that by having awareness of whether a package and we have this new package type attribute in the recipe so you can specify whether something is an application a share library a

static library a header only library by having a knowledge in the recipes themselves conan can make better guesses about what needs to be rebuilt based on changes somewhere in the dependency graph. Right. Within a local build system, one of the big challenges is when one build step produces artifacts that dictate what the next build step is going to be. And so you can't always sort of look ahead. Do you have that problem at the Conan scale? Absolutely. Yeah.

So, and the graph, again, helps in assistance as well. So Conan can figure when you have multiple kind of recipes from which you need to create Absolutely, yes. And the graph, again, helps in assistance as well. So Conan can figure when you have multiple recipes from which you need to create binary packages, Conan can give you the right build order in which they need to be built based exactly on this additional information. Absolutely. So now it models the intricacies of C and C++ libraries

and where shared libraries and where, you know, shared libraries behave and propagate different than static libraries, for instance. And now it gives you better tools of sort of either avoiding things like, you know, symbols. I think you mentioned this, Tim, earlier, having symbols in multiple places

that are incompatible with each other and things like that. Right. Yeah, so it's a lot of work you're doing on our behalf. Thanks for that. Yeah, thank you. The release notes also mentioned that you have a new public Python API. So what's that about?

When would I use the Python API to interact with Conan? Absolutely. So we do have a new public Python API that we didn't have before. So unfortunately, what you notice over the years is that people actually use your private API, and then that one can break, and people are not very happy about that. So now we have a public Python API.

One of the workflows that goes beyond doing a local build, so you have your own project, and you want to call con and install to bring the binaries for your dependencies. How does it do with continuous integration? So in continuous integration, developers are very likely

to have a script that drives this. So maybe they want a script that's already written in Python and they want to interact with Conan to drive the build. And maybe you need to call Conan to tell you what the build order is and orchestrate your CI

to build those things in the right order, maybe in parallel as well, if you can make that connection. Then you maybe want to upload those binaries to your Conan remote as well. So in Python, you can actually do this programmatically. We also have in Conan 2, all the commands now have a uniform

output as well. So we have JSON output. So optionally, you don't have to use the Python API if you can work with JSON output. You parse it and then you take the inputs you want. But to be honest, we'll discover, apart from the things that we're

aware of, we'll discover what people actually come up with, which is going to be really interesting for sure. so hang on conan is written in python isn't it absolutely yeah it's written in python ah that's cool so it's another another c++ tool that's actually not written in c++ seems like there's quite a few of them around so you know i think edg and gcc are written in c and And then obviously, like C-Line is written in Java and Kotlin. So I think there's probably more of these tools that are actually not written in C++. I know.

And this is something that we hear from time to time where some developers really want all of their tooling to be written in C++ so that they can sort of bootstrap it from scratch or something like that, which I suppose it's reasonable in some scenarios. But I always ask, are people really building? Maybe they are actually CMake from source and Git

and all these other things that they may need. So as far as I'm concerned, Conan is just an executable. The interface to Conan is just the Conan command and the command line interface. Well, and now, obviously, if you wish to do use, so the Python interface and parsing the output really in a machine-readable way. But once you get past the hurler of installing it, then it doesn't matter what it was written in. And we do have, and we make them publicly available as well, installers where the Conan executable is self-contained so you don't have to maintain a separate

like Python interpreter distribution or anything like that. So hopefully that makes it easier in those scenarios as well. Cool. Obviously you can write just about anything in C++, but it doesn't mean you should. Sometimes there's a better language for the job. Absolutely.

And to be fair, I suppose my argument for those people that really do want to build everything from scratch is, well, you can build the CPython interpreter from source, which is C, uses a compiler, and then use pip to bring content into a visual environment. Technically, I suppose it is possible. Well, there you go. So is there anything else exciting in this release that we haven't

talked about yet well we do have um as well and i think i i touched on this uh on this before the ability to have uh custom custom commands we see a lot of uh use especially for uh from enterprise environments that they have their own workflows it's like like, oh, I always call this command, followed by this command, or in my CI, I have this, followed by this, or the structure of my repository requires calling commands in a specific order. So now there's the ability for users and teams, actually, to write their own custom Conan CLI commands,

or subcommands, rather, and distribute them to the rest of your team so that everybody can use them. So that's actually very exciting. We started using this internally as well for some things in Conan Center as well. So the Conan Center repository is where we host all the recipes for the public libraries that we make available in Conan Center.

So it has a really well-defined structure. So we're using custom commands to actually automate or simplify some of the tasks. And it's quite exciting, to be honest. We really, really like the custom commands feature. It comes up a lot whenever we talk to developers who have not switched with Conan 2 yet and then describe workflows.

And we say, hey, you can implement your custom commands. It's probably going to be not as many lines as you think it's going to be, and it's going to make your life a lot easier as well. On the CMake integration side of things, and I think users will remember

the early days of Conan where you had to include a file that was generated by Conan, and then target link libraries against some specific Conan underscore package targets, if I remember correctly. One of the most requested features is to not do that. People, at least that we see, they don't want to change, make modifications to their CMake lists as scripts in order to be able to consume dependencies from Conan. So that's also been improved as well.

And now you have the option of generating a toolchain file that if you pass it to CME when you first configure, then CME will be able to resolve the find package calls to the dependencies that you have and also generate the targets with the same names that you were expecting rather than some arbitrary Conan ones as well. So that's actually been used at scale in Conan Central as well.

So we know it works quite well. Cool. So if I have a C++ library that's on GitHub, how do I get it onto Conan Central? How do I make it easy for people to get my library via Conan? Absolutely. So, well, you have two options. You can raise an issue with us in a Conan Center Index GitHub repository. Say, hey, I need some help

on making this available. Or if you want, you're more than welcome to open a pull request with the recipe itself. So it will run on our continuous integration service built for as many platforms

as we support. And then our community reviewers and our team members in the content team will make every effort to make sure it gets in. That's cool. So how many GitHub stars do I need for that? Not many, actually. We do have some checks

before allowing people to contribute. So we will flag things like just created accounts or something like that. So you do need to have some activity, but not even many. All right. Okay, well, we do have some more questions about package and dependency management in general, but we're going to come back to that after a quick sponsor break, which is going to focus more on the code actually, because this episode is sponsored again by Sona, home of clean code. And Sona Lint is a free plugin for your IDE, which helps you to find and fix bugs and security issues from the moment you start writing code. You can also add

Sona Cube or Sar Cloud to extend your CI CD pipeline and enable your whole team to deliver clean code consistently and efficiently on every check-in or pull request. Sonar Cloud is completely free for open source projects and integrates with all of the main cloud DevOps platforms. All right. Okay. Back to Luis. So we talked a lot about Conan and Conan 2.0 and how it all works and why it's great. I want to kind of zoom out a little bit and talk about the C++ community in general and kind of the ecosystem. So I guess kind of an open-ended question.

What do you think are currently the biggest challenges in this area of package management and dependency management, kind of more widely? Oh, that's a very interesting question because I could talk about this for hours if you ask me. I'll try to be brief on this topic. So I would say one of the biggest challenges, and I think we need to remind ourselves that C++ has been around for nearly 40 years now, right? If I'm not mistaken. There are no standard

ways to interface with the build system. There's no standard way to interface with a compiler other than the source code itself, but the command line flags every compiler implementation is free to do whatever they want. So that lack of centralization and sort of the passing of time means that there are many solutions to very similar, but not quite the same problem. And that diversity of things out there make it really difficult to solve the problem in a way that people like. What we see, especially from an accounting side, is that we get users requesting features

that are one thing and other users requesting exactly the opposite. So it's really difficult to actually satisfy the use cases. Or not even that, it's probably easy to satisfy the use cases if you just hack everything, right? But it's hard to do it with the right level of abstraction in a way that you can maintain over time. And I personally think that that's one of the strengths of Conan, is that it's allowing that diversity to exist while the C++ ecosystem continues to evolve. I think also something to keep in mind is that modern CMake, which

a lot of projects seem to be converging on, hasn't been around for that long. And now we're going to be adding modules into the mix. So I think it's safe to say that C++ is going to continue to evolve. But I think we need to start paying attention into trying to, I wouldn't like to say standardize, but maybe have better conventions for how to interact with a build system, maybe from the command line point of view, the compiler, or even agree on what's a library.

Depending on who you ask, you get very different questions about what a library is. Well, if I put my committee member hat on, there is no such thing as a library exactly exactly so it's it's difficult because a developer will expect maybe they will expect some developers will some developers uh uh won't um an abstraction right you're in cmake you do target link, my program or my library against something else. What is that something else? At the low level, that's something else.

It's just flags that get passed to the compiler, to the linker, maybe to the runtime linker to find shared libraries. But you don't want to be concerned about that. I've had developers in my previous team say, I don't care about this. No, it's like, it should be in a way that I don't have to like sort of dig through the the flags that are passed because of some other library and that's

true so where do you put the responsibility i think the responsibility is indeed in the package management side of things but then the package management side of things needs to be able to retrieve the information from somewhere and there are no ways for the library authors to express this information in a way that is also standard or are compatible across multiple systems as well so at the moment i think we're in a situation where the package repository maintainers are doing a lot of the network keeping the ecosystem sane and it's happened like for decades actually like at the linux systems have been doing it uh for a long time with obviously a lot of the network keeping the ecosystem sane. And it's happened for decades, actually. The Linux distros have been doing it for a long time

with, obviously, the limitations that those package managers only work either only for Linux or only for specific distros and don't work for macOS or Windows or other systems. But now we're seeing different things. But still, what I see is that there's a lot of work in maintaining or curating the package recipes or the packages themselves. So if we make more efforts in standardizing some of those things, I think a lot of the pressure of repository maintainers will be relieved as well. And maybe things become a lot easier to do so somewhere in your answer you

actually uh mentioned something that is a very good segue into my next question you mentioned modules so we got modules since it was 20 they were supposed to solve lots of problems but three years later i don't really see them being used uh in mainstream c++ development there are like i know that the CMake team has done a lot of effort to kind of try and make it work. There's like a few talks here and there where people who went through heroic efforts to like modularize their libraries,

but it's not that it's really a thing that's used more widely. I wonder if that has something to do with, yeah, kind of the impact that modules have just on what you have to do if you have a build system, if you have a package manager, if you have any of this plumbing around actually building code that just makes it so much harder. So how do modules impact you as somebody who works on a package manager or

where do you see like see like the problems kind of more widely absolutely from from what i remember using the module how to impose a requirement that your your compiler on your end needs to be able to invoke the compiler to i think create the module object well the equivalent of the object if i'm not not mistaken so as i mentioned before because there's no interface to the compiler that's imposed a new challenge is like where's the responsibility is the responsibility to the build system from i've seen you could argue the responsibilities in the compiler itself if it can actually work out because i think in order to resolve those imports it's going to have to locate some files and also compile them

in order to be able to produce code. So I think the answer is that we still obviously need to see this working without build systems being involved, if I'm honest. Like at the low level, how does it work and implementations? And then start building on top of that.

So I think CMake now is making a lot of effort in integrating modules as well. So I think that will be very good for us, especially from the package management side of things, to get visibility on and see, okay, what do we need to change about our model to fit the new world of C++ modules? I don't, to be honest, I don't know yet, because what's been exciting about following C++ modules. To be honest, I don't know yet because

what's been exciting about following C++ modules is that there were things that were unforeseen in a way that are now coming to the surface when things are being implemented. And they're not easy to solve. Again, part of the same problem of there's no standard way to invoke a compiler or interact with a build system. And now modules is forcing that a little bit, which I think is a good thing.

But it depends on what solutions we end up coming up with, to be honest. Right. So if I have my own open-source supersus library, you think it's probably a bit too early to put a lot of effort into modularizing it. If you haven't, you want to say go ahead.

If you don't today today then we're probably going to be using modules in like 2030 right we should like i'd say go for it that's optimistic um i suppose you may want to and this is a trade-off right you may want to um also allow your users to use uh your library without modules so that duplicates the effort so it's definitely a trade-off but i'm i'm always more like if it's if it's being approved um in in a standard um and it's the compiler has support i will use it done you know be be the change that you want to see yeah that's a good attitude i like I'm going to go and play with modules later today. You mentioned a couple of times that we need more standardization on some of these tooling aspects, but the problem is, of course,

the standard doesn't talk about any of those things, and they can't by design. But I know a few years ago, we we just set up the uh the tooling study group i forget which fg it is now i'm 15 i think yeah yeah that's the yeah i think that's the one one of the tasks they had early on was to produce an ecosystem report and i know some work went into that and i think i remember rightly it sort of fizzled out a bit just pre-pandemic. And I haven't kept track to see whether that's been picked up again yet. Do either of you know if anything's moved forward on that? I think there's a movement on that front and that there is a proposal indeed to have like a proper standard for ecosystem related things.

But, you know, things move slowly and i actually have to update myself on this right but absolutely i think when we went in cpp con uh last year there's definitely whispers of you know finally um we'll be moving along with this and i have seen uh that i think one of those documents about ecosystem and you know looking at that i say okay these are the good the right levels of of extraction right so what do we do next and one of the things that and again even even if you have like a really good like um standard that ticks all the boxes then next challenge is the community um actually implementing it and we do we still have lots of libraries that we see when you use auto

tools for instance and though those things um have not changed that much uh in in many many many years so it does beg the question is like how far can you get when there's like decades of legacy in in the c++ community and i suppose i't know. I try to remain optimistic, so as I always say, we'll see. But we have to try, right? So I can actually add maybe a little bit more to this. At the last couple of committee meetings, I actually dipped into

the SG15 meetings because I was just kind of curious what's going on there. So right now, they're aiming to write not just an ecosystem ecosystem technical report but actually come up with an ecosystem international standard so it's kind of still uh early days but there is like a paper um that's kind of setting out the goals and what they're trying to do it's p2656 you're going to add the link to the show notes. And they're kind of starting

with a bit more modest goals. They're not saying we're going to standardize the whole package management build system, everything. But they're saying, okay, let's standardize a minimum set of recognized file extensions. Because one problem is that, for example, especially

with modules, I don't think compilers agree on what the file endings are for binary module interfaces and C++ files and C++ module header units and whatever. So let's standardize that. Let's standardize some interoperation between build system and package manager. So I guess that's going to be relevant for you at some point.

So not standardize the interface to the actual build system or the package manager, but just some kind of meta language or something. It can be as simple as JSON or something with which the build system and the package manager can talk to each other. And let's standardize a portable diagnostics format that's i think the other thing that's looking into and um yeah so it's i think relatively

early days the last time i was in one of their meetings i didn't actually understand what was going on they were like talking about different compiler flags on visual studio and gcc for like four hours and how like this introduces like really subtle uh problems that somehow related to modules and i just really couldn't follow it to be honest but i think they were just like very deep in like a particular rabbit hole at the time but like um yeah this this paper um that's that's happening that's work that's going on so i'm actually really curious i think that that sounds like a good um first step forward absolutely i think um i'm very interested in exactly uh the bit that you describe which is the interface between um the build system and the package

manager it doesn't have to be the world we don't need to standardize everything but even um even just that from the consumer side of things, we do have really good solutions to some extent. If you're using CMake and the library, your consumer uses CMake as well, and it generates a CMake config file, then you're good. But what if it doesn't? What if it's using something else?

What if it's using the package config files? And how you do that in multiple systems, Linux, Mac OS, Windows, when things may be installed in different places and you may want to work with different versions of the same library across maybe different projects or even in the same project in multiple branches using different versions. So it's those things that I'm more interested in to see progress, to be honest, because that's going to solve a lot of the maintenance burden

that at the moment falls into the package repository maintenance. So I think it's fair to say there's a lot of challenges ahead. Oh, absolutely, yeah. In summary. Won't keep us busy. Before we round out the discussion on package management and Conan, I've actually saved the most important question for last. And that's, why is it called Conan?

Oh, do you know what? I don't remember. I think it's a play on Conan the Barbarian. So this is Conan the Librarian. I was surprised how few people actually get that one. Yeah, I remember there was a T-shirt at some point that you were giving away for a while with a giant humanoid frog

dressed up like Conan the Barbarian with a sword and everything. I still see a lot of people running around with those at conferences. It's a shame I never got one of those. If you ever have a new edition of those t-shirts, then I would be interested in having one because I think it's awesome. I'll see if we can find the one you're describing. We might have some left over. Because obviously I'm a big fan of Conan the Barbarian.

Yeah, I think I've got that t-shirt. I know, it's a good movie, right? Yeah. Right. So I guess I have one more question for you louis which is apart from all of the stuff like dependency managers package managers is there anything else in c++ that you think is really exciting um yes when i compare when i started using C++, which is about 2010, if I'm not mistaken, it's a lot easier now to use new versions of compilers, use new C++ standards, standard versions.

So there's like a cadence of every few years there's a new release. That used to be a nightmare. And if I'm allowed to use that word, companies would take years to move from one version of a compiler to to another um to move between uh versions of the c++ standard i think now it's a lot easier and it's not just one thing that's contributed to this is everything right the um there's a better cadence in the standards being released. The community is really good at using the new features, providing feedback.

I think there's a good process there when it comes to the standard itself. But the compilers as well, I like the healthy competition between GCC, Clang, and Visual Studio as well. And the package managers being part of the ecosystem and CMake having continued evolving, to be honest, and a lot of library authors choose CMake for that.

So I think it's a lot easier to use C++ these days, if I'm honest. So what's exciting is that that means that it's a lot easier to use the new features. And I think we all love at least... I mean, obviously, as you mentioned before, Connor has written in Python now, so I'm not directly writing C++ code myself. But it's like I obviously, as you mentioned before, Connor has written in Python now, so I'm not directly writing C++ code myself,

but it's like I said, I think if developers, whether they're in an enterprise environment or open source, if they want to use the new C++ features, it's a lot easier to do that. And there's a lot less sort of effort involved in that. But to me, that's exciting.

Just like the C++ today is basically a different language than if you look at the code that people are writing today, it's just completely different than it was 10 years ago. Yeah, with C++11, there's been a big influx

of new C++ developers who never really had to deal with C++ before C++11. And it's not always obvious that because there was effectively no standard between C++98 before C++11. And it's not always obvious that because there was effectively no standard between C++98 and C++11, there was 03, but that wasn't really a big change. So for 13 years, the standard didn't change effectively. And people got used to that stability and that became almost a defining characteristic of the language, both good and bad.

And then that would change overnight and it's become completely different but uh it's a very different world yeah completely interesting because i'm one of those people you just mentioned who joined uh kind of after all this i i started writing cybersus in 2010 if i remember correctly but before that i was just writing like c and fortran and so i started in 2010 at that point like css 11 was kind of already around it was called like css ox but like like gcc already had like the flag cpc or x and it already had like a lot of features so i kind of immediately started learning css 11 and i never really actually had to write css 98 code like ever so it's kind of interesting to hear that this it's been it's been a thing or it was a thing there was a very brief window where it was called

c++ 1x right right right oh yeah that's true yeah yeah yeah yeah that's kind of around the time when i started using it i I think. Right. Okay, so, Luis, anything else you want to tell us before we let you go? Not really, no. So, obviously, everybody's welcome to try out Conan 2.0, and we're very open to feedback, issues on GitHub. You can reach out to us on Slack and on multiple channels. So, by all means, like, Conan is made by a very small team in Madrid

and we're very happy to be a part of the community as well, but we wouldn't be able to do it without all the feedback we get from our users. So a big thanks to everybody using Conan and opening issues and helping us make it better

for other users as well. And a massive thanks to our open source contributors as well, both in Conan and Conan's end. Well, thank you so much for coming on and telling us all about Conan and Conan 2 and all of the issues around it.

Thank you for having me. Is there anywhere that people can reach you if they want to find out more? Absolutely, yeah. So for Conan, you can check out our Twitter account, which I think it's Conan underscore IO. And I'm jcar87 on GitHub. And I'm Julius on CPP Lang Slack, I think,

and many, many places as well. So we'll put those in the show notes as well. Perfect. No worries. So yeah, thank you. All right. Thank you so much, the show notes as well. Perfect. No worries. Yeah. Thank you. All right. Thank you so much, Lewis, for being our guest today.

Thank you. Thanks so much for listening in as we chat about C++. We'd love to hear what you think of the podcast. Please let us know if we're discussing the stuff you're interested in. Or if you have a suggestion for a guest or topic, we'd love to hear about that too. You can email all your thoughts to feedback at cppcast.com. We'd also appreciate it if you can follow CppCast on Twitter or Mastodon.

You can also follow me and Phil individually on Twitter or Mastodon. All those links, as well as the show notes, can be found on the podcast website at cppcast.com. The theme music for this episode was provided by podcastthemes.com.