CppCast - Conan

Episode Date: May 4, 2016

Rob and Jason are joined by Diego Rodriguez-Losada from Conan to discuss the new C++ Package Manager. Diego's passions are robotics and SW development. He has developed many years in C and C++ in the Industrial, Robotics and AI fields. He was also a University (tenure track) professor till 2012, when he quit academia to try to build a C/C++ dependency manager, co-founded startup biicode, since then mostly developing in Python. Now he is working as freelance and having fun with conan.io. News Robot: Native Cross Platform System Automation Help improve DuckDuckGo's C++ searches! Stay up to date with the Visual C++ tools on NuGet Diego Rodriguez-Losada @diegorlosada Diego Rodriguez-Losada's website Links Conan: C/C++ Package Manager Conan Blog I've Just Liberated My Modules

This episode of CppCast is sponsored by JetBrains, maker of excellent C++ developer tools including CLion, ReSharper for C++, and AppCode. Start your free evaluation today at jetbrains.com slash cppcast dash cpp. CppCast is also sponsored by CppCon, the annual week-long face-to-face gathering for the entire C++ community. Get your ticket now during early bird registration until July 1st. Episode 56 of CppCast with guest Diego Rodriguez-Lasada recorded May 4th, 2016. In this episode, we talk about automation with C++ and a new way to update the MSVC compiler. Then we talk to Diego Rodriguez-Lasada from Conan. Diego tells us about the new C++ Package Manager

and some of its key features. Welcome to episode 56 of CppCast, the only podcast for C++ developers by C++ developers. I'm your host, Rob Irving, joined by my co-host, Jason Turner. Jason, how are you doing today? Doing all right, Rob. How about you? Doing pretty good. It's Star Wars. Happy May the 4th. May the 4th, yes. It's always a great holiday in my house. Do you take the day off, put up ribbons and everything? No, no, but I probably will watch the newest Star Wars movie this week.

Which you have already seen, right? Oh, yeah. I saw it in theaters. I got it on Blu-ray in the morning. Are you ready for C++ now next week jason um let's say 75 percent ready 75 percent you still got a couple more days yeah and i've got six days until my first presentation which is the one that i'm the most ready for so right should be all right. And we will be taking the show off next week and we'll return the week after C++ now. Right.

At the top of your episode, I'd like to read a piece of feedback. This week, I got a couple of tweets, or the last two weeks, actually. One from Renaud Lepage and one from Dave asking when CppCast is going to be on Google Play Music. I didn't really hear about this until they reached out to us on Twitter, but I guess Google Play is having their own podcast subscription service built into the Google Play Music platform now. So CppCast is now available there. Cool. Yeah. There's subscription links on the CppCast website for Google Play, and I tweeted and Facebook posted those links out yesterday.

So if you use an Android phone, it's probably a really good way to get the podcast on your phone. Yeah. Well, we'd love to hear your thoughts about the show. You can always reach out to us on Facebook, Twitter, or email us at feedback at cppcast.com. And don't forget to leave us reviews on iTunes as well. robotics and AI fields. He was also a university professor until 2012 when he quit academia to build a C++ dependency manager and he co-founded the startup Bcode, since then mostly developing in Python. Now he's working as a freelance and having fun with Conan.io. Diego, welcome to the

show. Thanks, Rob. How are you? Hi, Jason. Hey, Rob. I'm great. Diego. Okay. Before we continue into the news, I want to ask about this. It had to have been a huge decision to quit out of the tenure track for university professorship to go and do a startup. Yes, it was a huge decision. And even because finally we had to close it, but I don't regret anything. It was the best decision I could make. I'm really happy with it and how it's going. So what were you actually teaching? Were you in the computer science department or

robotics? I was teaching in the electronics, informatics and automation department and I was teaching in the electronics, informatics and automation department and I was teaching mostly software engineering, teaching C and C++ and some automation also and robotics in the PhD level. But yeah, in the degree level,

bachelor level, I was teaching software engineering the most. Wow. It's a big decision. Yes, but I mean, there was a huge crisis here in Spain. When I took the decision, many people told me that I was kind of crazy. But you know, you cannot complain about the crisis and do nothing. So finally, I had the opportunity.

I had the venture capital to back up the startup and hire people. So I felt like a kind of responsibility to do it. So yeah, and totally the right thing to do. Wow, that's cool. Thank you. Speaking of robotics, this first article we have is actually called Robot, although it's not really related to robotics. It's a native cross-platform system automation library in C++.

And I guess it is used for doing keyboard loggers, things like that. I'm curious what your thoughts are on this, guys. I'm not really sure. I tried yesterday. It is more like to create events. You can create keypress

events, mouse events. You can capture the handles of the windows. You can automate, for example, testing of a GUI application at the system level. So it's a very nice automation tool, especially because it is portable for the three major systems. So it's more or less what it does.

It seems like maybe something that could be used for building automated testing frameworks for your GUI-based applications from the way it's described? Yeah, totally. Okay. Okay, that does sound pretty powerful then. Yeah.

Yesterday I was having a look and I said it's like three libraries in one because it has completely different implementation, of course. It's very low level. So for Mac and Linux and Windows, totally different implementation. And I check also other alternatives, and it seems like the best thing,

if you want to do that kind of things, is probably the library to start with. Okay. Well, it's definitely worth checking out then. Jason, do you want to introduce this next one? Yeah, so I use DuckDuckGo mostly for my searches, and DuckDuckGo has

sent out a call saying that they want help so that people who search on C++ topics get better results back. So they've got cheat sheets and stuff that can be updated for C++ searches. And I just thought it was cool

and something the community should be checking out to see if we can help improve the search engine. Yep, very cool. This next one is from the Visual C++ blog. And we've talked a bit about how they now are able to deliver the C++ compiler kind of as a separate install. You no longer need Visual Studio. If you want to just get the compiler, you can just get that instead.

And I guess as part of that, they're now making it so you could get a compiler update outside of the major Visual Studio update releases, which come about for a year. So it comes as a NuGet package. If you don't want to wait until Visual Studio 2015 update three, you might be able to get a compiler update through NuGet instead, which is pretty nice. That's neat for those of us who like playing with compilers. Yeah. Yeah. I'm curious as to how often they're going to be updating this. I don't know. I don't think it really goes into too much detail. Actually, we've been working on increasing the frequency of integrations,

but it's currently about every 10 to 14 days. Still, that's pretty impressive to potentially get a compiler update every two weeks. Almost a little scary, you might say. Yeah, a little bit. Okay, Diego, let's start talking about Conan. First, am I pronouncing that right? Is it Conan, Conan? Conan, I think.

Conan, okay. Conan. Can you give us an overview of Conan and how it's used? Yeah, sure. Now, Conan is a free and open source project, MIT license, that is implemented in a CNC++ package manager. It has two parts, a client application and a server application. Because it is fully decentralized, it is like Git-like style. You can have many remotes and you can run also your own server in-house easily.

An important thing about Conan is that it handles both building from sources and also binaries with many different configurations. We like to play with different compilers and options and 32-bits and then shared libraries,

static libraries. So it will handle any configuration you want. And finally, an important thing is that it is build system agnostic. It doesn't depend on any build system. It is able to wrap any build system. So it's quite like an orthogonal component. It's an independent component from build system. So it's quite like an orthogonal component, an independent component from build

systems. So how is Conan related to your previous project, B-Code? They had similar goals, right? Yeah, the goal actually is exactly the same. B-Code was an amazing idea but you know from my past I was a professor probably it was a bit academic and opinionated so yeah it had some problems to scale you know C++ projects are really large on average so you cannot build from sources always and that was what Bicode did. So it had problems to go to corporations

and being adopted in a wider scale. So it was a venture capital startup. We had to close it. But, well, we had learned many lessons and we still strongly believe that a package manager would be one of the best things for us as a community of C++ developers.

So basically we didn't give up. And me together with Lasso, he was the former CTO of vCode, we started to think in a new solution. It is totally from scratch. I mean, we took all the

lessons and start again. So technically the solution is completely different. They don't share a single line of code. They address the same problem. Okay, so dependency management in C and C++. But the overall approach is totally different. So yeah, it is related because we came from there, but technically it's just a different solution for the same problem. Could you go over what some of those key differences are between B code and Conan, and why Conan is an improvement over what B code provided?

Yes, basically what I said in the beginning, for example, being able to handle binaries. This is very important because we know that it was really requested in B code. Hey, can you handle binaries? I'm having a project with 50 or 100 libraries and I cannot build them from source always. So handle binaries is very important. And also we realized that because it was based

on CMake, we realized and we know that probably 50% of the projects they are using CMake. And you cannot restrict yourself to 50% of the projects in C++. You have to address the whole community if you want to go far.

So what we did in Conan is just not to be build system dependent, but independent. So for example, we now are able to build packets from Golang or just handle binaries without sources. So another important thing is to be decentralized. We have many requests to be,

oh, I want my own in-house server. I cannot depend on outside computers. So it requires a totally new approach, you know, like Git, like being able to do remotes, get the artifacts from one remote and then upload them to your own remote because you want to be safe and in-house.

And it has to be designed from scratch, from the origin. So those are the main key differences. Also, the usability is one order of magnitude easier because now we are using Python recipes that are inspiring

in OS X Homebrew but written in Python. So building a package is much simpler. You just write a Python recipe that wraps your build, any build CMake or whatever, and it will define like source, build, package methods in that recipe, and that's all. So from the usability point of view, it is much simpler. For example, you know the robot library we were talking about before?

I did a test yesterday, and for example, example in 25 minutes approximately I was able to create a package for Visual Studio 12 to 14, sharded and static libraries, parents and also a package test that it was able to test that the package was correctly created and everything in our repository and uploaded to the Conan.io repository. So it's not just me because I'm an expert in Conan, but we have been reported like 10 times improvement in time of creating packages. So the usability and the easy to create packages is also much much improver

in Conan. Okay, so you said it took you maybe about 25 minutes to build the recipes and build the packages and everything for robot. Now, what does it look like from the person who wants to actually use those packages? Yes. For the

people using the packages, we wanted to make it even simpler. So you have a text file. In that text file, you declare two things. First, the list of requirements, your dependencies. And then you define, we have the concept of generators. So you define one of our built-in generators like CMake, Xcode, Visual Studio, Premake, one of them.

And the system will just generate a file for you. In that file, you will have include paths, library names, CPPB flags, linker flags you need to use. But it is not intrusive at all.

You just use that file. If you are in CMake, you include the file. If you are in Visual Studio, you load the Visual Studio properties file inside your IDE. And that's the user experience. Just

Conan install that uses a Conan file.txt Conan install that uses a Conan file.txt and then it creates a file for you. You use that file in your build system. So it is quite convenient. Wow. And everything

is cached locally. So the projects if you have to build them or if you have to retrieve boost, for example, it's going to be a large download. Everything is cached locally. So if you use the project, that dependency again, is going to be instantaneous in another project or in another build. So everything, the local cache is also very important. It allows to be fast while using packages, and also to create packages locally

without requiring any remote at all. So you test everything locally, and then if you want, you just upload or retrieve packages from remotes. So if the package exists on the remote, it grabs it. If it doesn't, it builds it locally, and it caches

it in either case. Yeah, absolutely. We have the concept of package recipe and then we have the concept of package binary. So when you declare a requirement, you get the recipe and you check then your settings.

If you are using Visual Studio 14, this flags the static library, that will define a hash, an ID of the binary package you need to link with. If that binary package exists in the remote, then it will retrieve the binary package. Otherwise, you can opt in to build the package from sources if you want. And once it is built locally,

you can also upload to your own remote if you want. So you can cache it locally or cache it in-house in your server. So yeah, that is more or less the way it works. So it kind of sounds like if a user wanted to, they could almost use this as just simply a very high level,

uh, like a C cache just for caching their own local builds. Yes, yes, absolutely. Okay. Just curiosity. Make sure I understand the project.

I'd like to interrupt the discussion for just a moment to bring you a word from our sponsors. C-Line is a cross-platform IDE for C and C++ from JetBrains. It relies on the well-known CMake build system and offers lots of goodies and smartness that can make your life a lot easier. CLion natively supports C and C++, including C++11 standard, libc++, and Boost. You can instantly navigate to a symbol's declaration or usages too. And whenever you use C-Line's code refactorings, you can be sure your changes are applied safely throughout the whole code base. Perform unit testing with ease as C-Line integrates with Google Test,

one of the most popular C-plus plus testing frameworks, and install one of the dozens of plugins like Vim emulation mode or Go language support. Download the trial version and learn more at jb.gg slash cppcast dash cline. So Conan's only been around for, I guess, five or six months. Is that right? What kind of response have you been getting so far? Yeah, it was released in December, so that makes five months, right? Yeah, it was released in December, so that makes five months, right?

Yeah, five months. So if we are talking about metrics, then we are having like downloads, like let's say like a few thousand downloads per month. I would say like maybe two, three thousand. It's difficult to know because, you know, PyP, the Python Package Index, it has a lot of noise. It has mirrors.

And so it's very difficult to know the exact number. But let's say about 2,000, 3,000 downloads per month. It is pretty good. But, you know, this is a vanity metric. So we are actually measuring the API calls that the Conan.io server is getting

and they are increasing like 25% month over month since we released it. And we have reached like almost 100,000 API calls last month. That is pretty good. I mean, the real activity reflected in our server is increasing,

and it's quite good. I would say that one order of magnitude better than we got with B code. So we are very happy with this. But if you talk economically, it depends. For example, we have been reached out for investment. So there is interest from venture capital to invest in this idea, in this project. But also we have already had some paying customers.

We are happy about this because it was kind of natural. You know, with B code, we had to do a lot of sales and try to convince people. And in this case, they just came. Hey, this is cool. This is working fine. We need some help. Can you give us that help?

Okay, so our revenues are already higher than B code. That's good. But the two most important things that we are happy about is we know that there are a few companies that are actually

using it in production. It is not stable yet, but they are using it, and they are happy about that. And also that the community is working very fine. We are like 12 contributors in the GitHub

repository. That's a lot. They are helping a lot with issues, reporting things, pull requests, generators. So the community is quite involved. That didn't happen either in vCode. So I would say that this is a good sign that this time is going to be the good one.

So yes, overall, the response is great, and we are really happy about it. That's great. Without digging into the economics too much I guess if you are maybe an open source contributor you can use Conan for free but if you want to use it for

your company is that when you would pay to get your own Conan server? Is that about right? No, no, no. The Conan server is also free and open source. Actually those companies they are using the in-house Conan server is also free and open source. So actually those companies, they are using the in-house Conan server for free. Okay.

Yeah, it's totally free. You can use the full stack free. We will support in the future premium accounts in the Conan.io server. And now what we are doing is consulting. So companies are paying us to help them to bootstrap their projects. For example,

we have these dependencies. What is the best way to arrange the projects, to manage the dependencies, to split things and so on. We are helping them just preparing their projects. So yeah,

consultancy. We don't have premium accounts yet. So our business model is still not defined. But we don't care about it. Now we are free. We are not

venture capital backed. So we can decide care about it. I mean, now we are free. We are not venture capital backed. So we can decide where we go. So for now, our priority is people using it. And if it has to be free, it will be free forever, MIT license. So this is the first principle we have. And then if we can make money and people hire us for training or consultancy, or they want their premium accounts

in Conan.io, we will do it. But that's a secondary goal. Okay. Sounds great. Thanks. So you said the API, or that the project hasn't reached a, quote, stable point yet. Do you know when that's

coming or how much more work you have to do to get there? Yes, it is in our roadmap about September, September, October, it depends on many things because the user API is

quite stable right now because as I told you, there are people using it, so we are very careful not to break things. If we introduce changes, then we also implement automatic migrations. And also, so we are

in general very careful about breaking things. If we break things, they are really minor things. So, so far, the user API is quite stable. It is true that the server API might change widely in the next months, but that is not a problem because users are not actually connecting to the server in a row. You know, they're using the user API and the client application. So for the users,

it is already very stable. So we think that in a few iterations, maybe two, three releases more, and in September, we will go 1.0. It depends on other factors, of course. I mean, it's not just an API thing, but we think that when we go 1.0,

we have also to provide some kind of better support, be able to support 24-7 and things like that. So it will be kind of API stability and also maybe business. At some point, we will have to do something because we are paying now the servers and everything. So 1.0, September, October, are related to API stability

and also probably some business movement. Okay, so since we're going into versions, you actually just announced version 0.9 was released yesterday. Do you want to tell us about some of the new features that came with 0.9? Yes, sure. For example, one of the things we did, we implemented Python 3 support.

You know, all the modern distros, now they are coming with Python 3 as default. And we're using Python 2 as the default. So we have added Python 3 support for the Conan code base. It was like an internal change, but necessary to adapt to the new Linux distros. An important change also was that we are using now, like, how to say, generator packages.

So if you want your own build system, your own generator, you can create it. And you can create it like a package. So you just write a package recipe, and in that recipe, you write your generator. So actually, the pre-make for generator is built that way. So this is a major change because now you can upload and share your generators, you can write a generator for any tools you want, you can version your generator so you don't

break things. If you improve your generator, for example for premake, it's not a problem. Just generate a new version of the generator, and you will be fine. So this is an important change because it is like a plug-in mechanism. So it allows the community to create their own tools around Conan. So I'm curious. You've mentioned several different generators.

What build systems do you officially support? We have generators for CMake of course it is our most used one we have a generator for GCC you can use it too we have for Xcode we have for Visual Studio we have other tools too,

like the autocomplete me for Vim. Oh. Yeah. Anything that takes your input and your data and generates a file for any of your tools, you can do it. And I would say some, then we have like QBS and QMake also,

generators for Qt Creator. And I would say those are the build systems officially supported right now. But as I told you, many of them can be developed right now. So actually, if there is interest, we will help to create a new generator for that build system. Okay. I want to shift gears a little bit

and ask you about a different package manager, Node, and a bit of a debacle they had a couple weeks ago with LeftPad. And I was wondering if you could maybe tell our listeners about it, for those who don't follow the JavaScript community, and then maybe we could go into whether you

think Conan should be kind of safe from something like that happening. Yeah, sure. What it happened is there was a developer, kind of safe from something like that happening. Yeah, sure. What happened is there was a developer, Acer, I don't know exactly how to pronounce that name, that after a dispute

over the name of a package, the package was called Kik, then the company called Kik, they won the dispute, so the NPM company, they decided they won the dispute, so the NPM company, they decided to remove the package, not remove, but give the package name to the Kik company.

So that developer was upset with a reason, I think. He was upset, so he decided to remove, to unpublish all his packages from NPM server. And he had like, I don't know, maybe 200 packages? 200 or 300 packages? Yeah, yeah, like 200, 300

packages. So he removed all of them. And the problem is that some of them were really important, like the left path. It was like a small package, very small package, like, I don't know, maybe 15 lines of code or so. But thousands of other packages were dependent on this particular package. So, the moment he removed the package, he broke

like thousands of builds in the world. I think it was not too long, because it was like two hours and a half. And active websites, right? Like, active websites were broken for a little bit. Is that correct? Yeah.

Or was it just the builds? Okay, I'm not sure. I think it was just the people who were working in development environments and trying to grab at that moment the latest package. Yeah, yeah. Actually, yeah, it was like breaking the builds, not

systems in production. It was breaking the builds of many, many thousand developers. Then they were lucky because the community stepped in and they started to create replacement packages

for those names because the problem with NPM is that they have global names for packages. You can have the map package and this is going to be a name for a package in NPM. Then, if you decide to remove

the map package, then another developer can come and publish a package with the same name. And especially because NPM has like version ranges, and so you can depend on any version of that package. So if a new developer publish malware in those packages,

then the infection could spread like, whoa, like incredibly. But they were lucky because the community, their reaction and in a few hours, they were replacing the packages. But the security hole that was exposed several hours was really huge. So the concern

there is actually... The problem is that people are using NPM for deployments. The NPM developers use it only for development, not for deploys. But yeah, it's very convenient for deploys.

So actually people do what is more convenient, not what they are told to do. Yeah. So that's the way we developers work sometimes. So yeah, that was a problem.

And regarding, can this happen with Conan? Well, I think no, this cannot happen. It is true that we allow deleting, removing packages. There is a lot of controversy. Should a

package manager allow, for example, Java, Java Maven Central, they don't allow removing packages. Once you upload something, it's going to be there forever. So there is quite a discussion if you should be able to remove or not. We allow removing because we think that developers

and package creators, they have the right to remove their package if they want. But what is Conan doing to avoid this NPM disaster? So, you know, in C++, we love namespaces. So we are using namespaces. So basically, every package is under the username. So the package references are actually the package name, the package version, the username, and the channel. Because you might want to publish a package under the testing, stable, or whatever thing you want.

So once the package references are using namespaces and the namespace is the username, then you are absolutely safe because no other user can replace the packages that a user has created. Maybe removed, but

there is no name classes, name collisions will never happen. This is also something we want because you can have your own package for Boost, for example, if you want. And I want to have my own package for Boost. How can this exist

without naming Boost, Rob, Boost, Diego, Boost, Jason? Then the package is called Boost and then it's under the name space of your username. Rob, Boost, Diego, Boost, Jason. Then the package is called Boost, and then it's under the namespace of your username. And this allows also to, in Conan, you can fork. You can copy dependencies. So, for example, I can get a Boost package from any source. Then I can copy it under my username, and then I can upload it

with my username to my own server. So then I'm totally safe. I could disconnect from the Conan.io server, and I will have my safe in-house remote, and I will be totally independent. So namespaces, forking dependencies, and being decentralized, I think this is more than enough to be totally safe. Finally, we also support like we have manifest. We hash every file in every Conan recipe and in every package. So you have the manifest with all the hashes of every file. So if you want to use that

to ensure that your dependencies are not being faked or substituted for another thing, you can just get the files and compare the hashes and you will be safe that your dependencies are not being replaced. So I would say that Conan is quite safe regarding this potential problem. Okay. Diego, is there anything else you want to go over before we let you go? Not really.

Just thanks again to all of our users and the community because they are really are really helping us to improve Conan and to develop Conan and we couldn't make it without them so just my best thank you to all of you Okay and where can people

find more information about you and more information about Conan online? Where about me my twitter handle is Diego R-L-O-S-A-D-A Diego

it's difficult but search for Conan.io and there you would be able to navigate from there to all the documentation, the packages and everything. So for Conan, Conan.io is going to be, everything will be there. And for me, just Google Diego Rodriguez Lozada and I think I will be the first occurrence in Google.

Okay. Thank you so much for your time today, Diego. Thanks very much, you, Rob, Diego. Thanks very much, you, Rob, and Jason. Thanks for joining us. You're welcome. Thanks so much for listening as we chat about C++. I'd love to hear

what you think of the podcast. Please let me know if we're discussing the stuff you're interested in or if you have a suggestion for a topic. I'd love to hear that also. You can email all your thoughts

to feedback at cppcast.com. I'd also appreciate if you can follow CppCast on Twitter and like CppCast on Facebook. And of course, you can find all that info and the show notes on the podcast website at cppcast.com. Theme music for this episode is provided by podcastthemes.com.