CppCast - ChaiScript and Cross Platform C++

Episode Date: March 5, 2015

Episode number 2 of CppCast with guest Jason Turner Jason has been developing portable C++ since 2002. With very few exceptions, every line of code he has written since then has had to run on multiple platforms. He is an independent contractor focusing on cross-platform issues, utilization of C++ libraries from scripting languages and code quality assurance. He is the co-creator and maintainer of ChaiScript, a mature scripting language designed for modern C++. His latest project is cppbestpractices.com: a fledgling effort to gather the collective wisdom of the C++ community. News CppCheck Four things you probably didn't know about C++ Boost libraries are now supported in biicode Jason Turner @lefticus Github EmptyCrate Links ChaiScript CppBestPractices Complex Object Initialization Optimization with IIFE in C++11 C++Now!

Episode number two of CppCast with guest Jason Turner. In this episode, we talk about a few things you probably didn't know about C++. Then I'll interview Jason Turner about the benefits of cross-platform development. And we'll also talk about ChaiScript only podcast for C++ developers by C++ developers. I'm your host, Rob Irving, and I wanted to start out this episode by thanking John Kolb for being a phenomenal guest for the first episode. I think we had a very informative discussion about the state of C++ and the C++ developer community. And I also wanted to thank all of you who tuned into that first episode. I was pretty blown away by the amount of listeners that the first episode reached. So again, thank you. And at the same time, please help me spread the word about CppCast to even more C++ developers.

Tell your coworkers, tell your friends at user groups, and if you use iTunes, please take a moment to rate the show because that can really help me reach more listeners. Joining me tonight is Jason Turner. How are you doing, Jason? Good, how are you doing? I'm doing great. Jason Turner has been developing portable C++ since 2002. With very few exceptions, every line of code he has

written since then has had to run on multiple platforms. He's an independent contractor focusing on cross-platform issues, utilization of C++ libraries from scripting languages, and code quality assurance. He is the co-creator and maintainer of KaiScript, a mature scripting language designed for modern C++. His latest project is cppbestpractices.com, a fledgling effort to gather the collective wisdom of the C++ community. Jason, thank you for joining me on the show.

No problem. Glad to join you. Yeah. So I just had a few news articles that I thought we could talk about. These were kind of pulled from Reddit, r slash cpp, isocpp.org, or just other C++ news sites that I follow. The first one is a new tool called CPP Check, which is kind of interesting. It looks like it's a static analyzer that's made to be extremely user-friendly. Have you taken a look at this, Jason? Yeah, I've actually used CPP Check fairly extensively for my own projects and for projects that I work on with my clients. It's a great tool for pointing out semantic errors in your code. Yeah, it seems like its real selling point

is that it's just really easy to use. It has a nice, friendly GUI interface, and you can just kind of point it at your folder with all your source code, and it'll start looking for errors. Is that about right? Yeah, that sounds about right. I've only used the command line tool actually myself.

I haven't used the GUI yet. But I use it in automated fashions for our continuous integration environment. Right, so if you want to use it in a command line fashion or hook it into existing build tools, it looks like it has a lot of support for that. Visual Studio, Eclipse, Jenkins, Hudson, plugins, Taurus SVN. So it really looks like it's a pretty powerful little tool. Yes. Okay. So the next thing I want to talk about was this interesting blog I found, blog article titled, Four Things You Probably Didn't Know About C++.

And it was just a list of things that are in the language that I know I wasn't aware of up until reading this article, and some of them are just kind of interesting. Did you take a look at this one too, Jason? I did. The main thing that really stood out to me on that was the tricks with ternary operators that he points out. I had never seen that before. Yeah, so that's the second one down, assigning to a ternary statement. Of course, you're used to seeing ternary statements as, I want to assign A to either B or C. So you'll say A equals B greater than C, and then either B or C, depending on how that condition evaluates. But apparently, you can also assign to a ternary statement.

So you could say A is greater than C, BA equals 5, something like that. I'm not really sure why you would want to do that, but it's something you can do. And also you can even evaluate a function on a ternary statement. So if you have two person objects, you can say if A's age is greater than B's age, then A or B dot print age, something like that. Pretty interesting. Yeah, that's pretty crazy. Yeah. The one that kind of blew me the most, actually, was the array operator being associative, just because it looks so weird. But basically, the point of this one was that, you know, because a standard C-style array is just using pointer arithmetic to get to like the second index of a five value array

you know typically you would say you know array bracket two if you want to get to the second item but you could also say two bracket and then the name of your array and you wind up getting the same value yeah that's i saw that and it also surprised me, but I thought, I have no practical use for this trick, so I pretty much glossed over it. Yeah, I think the blog post actually started off by saying that this is by no means things that you should be doing in your code. There's no practical function to most of this,

but it's just kind of neat to see some of the ways that C++ works that you would never really think of. The last one, which might be a little more applicable, was about pure virtual functions with a function body. And normally, when you have an abstract class with some pure virtual methods you would think that you're just using that to force yourself to implement that method in your derived class but if you do want to have some common functionality in that base class that every derived class is going to use then you can go ahead and you know define define the function body for that virtual method. And from the drive class, you can then call that, which is pretty interesting.

And then this last article I have is boost libraries are now supported in Bytecode. Now, I'll be honest that I'm not too familiar with Bytecode, but it is a C++ dependency manager, which is something that it seems like C++ could definitely benefit from. And they recently added support for Boost. So if you want to get Boost through Bytecode, that is something that you can now do. Are you familiar with ByCode at all, Jason? I've heard of it, and I followed some of the news about it recently, but I haven't used it myself yet.

I have been following when I see news about ByCode, and also recently I saw some news about Nuget supporting Boost also, which is a visual studio dependency manager of some sort. I feel like this is one of the, probably the main thing that C++ has a disadvantage over the scripting languages.

You've got Ruby gems and you've got the Python's dependency tools. It's awesome to see the community moving in this direction. Maybe we'll start to gain some ground and make it easier to share code and share libraries. Yeah, and that definitely seems to be kind of their mission statement at ByCode to give those kinds of tools to C++ developers. And if you ever need to switch between versions of a different library that your code is depending on. They're trying to make that very easy using by code. So pretty neat stuff.

I definitely recommend all listeners checking it out and seeing if it's something that they might want to use. I definitely find it interesting, yes. So let's go on to you, Jason. Um, I heard that you are going to be speaking at this year's C plus plus now conference on cross-platform development. Could you give us a bit of a preview for that talk? Sure. Uh, the talk is called thinking portable. And my main goal for this talk is to convince you that if you're currently working in a monoculture environment, only developing on one platform, that you should support multiple platforms, even if you don't have a business need for it today, that by supporting multiple platforms, as early as you can in your project's lifetime, you will gain the benefit of having more tools at your disposal

to help you develop better code and making sure that your code is cleaner and you don't leak in any operating system-specific or compiler-specific idioms or anything. Okay, so are you saying multiple platforms know, multiple platforms or maybe just starting off with multiple compilers? Like if you're living in the Windows world with Visual Studio, you're obviously using MSVC to do all your compiling, but maybe you would gain some benefits just by using Clang or GCC in parallel. Right, if you want to, yeah, if you want to boil

it down to that, then I would say multiple compilers is specifically my point. Although multiple platforms gets you there by necessity, really. So say if you're using GCC on Linux and MinGW on Windows, you haven't really gained the benefit of having more tools widely available to you. So you're right, I'm suggesting specifically multiple compilers across, well, presumably across multiple platforms, because then you also don't get stuck in any ruts with operating system specific code leaking throughout your entire code base. And also, if you're just sticking with Clang and MinGW and MSVC on Windows, you still might get yourself stuck in a rut of using operating specific function

calls. Some of the benefit of doing cross platform development is that you have to think about your code separation and think about what's your core API and, and think about where the operating system specific code needs to live. Right. So I guess part of what you're saying is, you know, if you're working on just one platform, you know, you might have the Win32, CAPI scattered all over your code. But if you start focusing on multiple platforms, then you'll learn, well, it could really be better if I put all of this in kind of one abstraction layer. And then if I need to go to another platform, I can just rewrite that abstraction layer for Linux per se. Or you might get yourself stuck with it. You might accidentally be using MSVC specific data types like D word or whatever throughout your code base. And if you're supporting multiple compilers

and multiple operating systems, then you really distill your code down to the cleanest, most portable, simplest code, I believe. Okay, that sounds like a great talk. What are some of the most common issues you encounter when working on cross-platform C++ code base? I think the obvious issues of GUI libraries being different, that's something that most people would think about right off the top,

is clearly if you're developing on X Windows or if you're developing on Microsoft Windows, you have a completely different set of GUI APIs. But I think the least obvious thing that I'd like to mention right now is file system access, which we think the IO stream libraries are class platform, and that's true. But being able to manipulate paths and concatenate paths and find directories and folders and stuff, that's something that's not obvious and something that currently not a lot of libraries do a great job of handling.

Qt, if you use Qt as your file system abstraction layer, does a good job. Boost file system has some surprising lingering issues on Windows of not working with path names that are over 256 characters long or something like that. So that's just maybe a little off topic, but the people that I work with are all looking forward

to the C++ standard having a file system library in it at some point here, which I think is hopefully planned for 2017, or they're discussing it, I thought. I'm not familiar with that, but that would definitely be very welcome to anyone working in this type of scenario. So you're actually signed up to give two talks, though, at C++ now, the second one being about IIFE. What is IIFE exactly? IIFE stands for Immediately Invoked Function Expression. It is the technique of defining an anonymous function and calling it at the same time.

So this function is throwaway. There's no way to reuse it because you've never given it a name. It is used in JavaScript extensively, which is where the name came from, because it defines a new context for variables in JavaScript. We don't really have a use for that in C++. But in C++, it gives us the ability to clean up object construction. So if you've got an object that doesn't have a well-defined constructor or has multiple steps that you need to use to construct the object,

this can give you a way of wrapping up the creation of an object and creating your local variable without having any stray data in your function and any other variables that you don't need lying around or it can also increase performance and make your code safer in some cases. Yeah, so I actually read your blog post where you first went over this concept of IAFE with C++, and I saw your notes on the performance improvements. Can you go over that in a little more detail? Yeah, it's kind of surprising, actually.

You can get almost exactly the same performance improvements by taking advantage of return value optimization in C++. So if you had, like I said, an object that took multiple steps to construct for whatever reason, you're using a poorly developed library, or you're just doing something complicated, you could define a function that creates your object and then returns it back to you. A normal named C function. It doesn't matter. And the compiler is able to just return back the object that was created using return value optimization,

and virtually all compilers are very good at doing that. I think, I guess I don't know the standard well enough, but I believe the standard either requires it or specifically allows return value optimization. And so you don't have, if it's an object that took multiple steps to create, you might have to create it first undefined and then define it and you're going to have an object that has no use in the first place, and then you have to do an assignment or a copy or something else into it, and you're taking the cost of the extra assignment or the copy or the second object construction. If you have a function to find the object for you and return it,

then you get to do that in one step. You take advantage of return value optimization, and you save the cost of the copy or the assignment. So it's a long way around of getting to that point. And doing that with IAFE makes it into a much smaller block of code. You don't have a function that only has a single purpose lying around there. But the surprising part is that after I posted my article to Reddit, some guys there took it up and they ran a bunch of their own tests to see

what performance differences they could see. And IIFE was faster in almost every case than calling a named function. And I don't know why yet. That's one of the things I'm going to research before I give my talk to make sure I can give some specific examples of what compilers are doing there. It doesn't really make any sense. It shouldn't have been faster. The compiler should be able to do the same thing. The only thing I can assume is that the compiler is able to do a better inlining of the object creation

and the function call there. Okay. Well, I will definitely encourage all listeners to check out this article. It sounds like it's a great way to both get a little more performance out of your code while at the same time having more readable code. Thank you. I think so. Great. So one of the really interesting things that you work on, at least I find interesting, is a project called ChaiScript, which you are the co-creator of. It's an embedded scripting language for C++. So I've used scripting languages a lot for various automation tasks. I use a lot of Ruby,

but I never thought to embed a scripting language into C++. What is something that you would do with ChiaScript? Could you tell us a little bit more about the project? Okay, so with scripting in your C++, you can get the best of both worlds of having flexibility for runtime configuration of your program while maintaining the performance characteristics of C++, or at least some of the performance characteristics. You're going to have, obviously, the overhead of calling

between the ChaiScript or scripting layer and your C++ layer. So ChaiScript is... The idea was to make it as easy as possible to add this kind of runtime configuration, runtime scripting into your project as possible. And I believe we've accomplished that. You can include just a couple of header files and execute a couple of statements

and be wrapping your C++ code in a way that it can be called from our script environment. Okay, so when you talk about runtime scripting, does that mean you can modify the Chai script code without having to recompile your C++? Sure. You could reload your ChaiScript, or you could modify it, re-execute it, whatever it's, or load it dynamically from the file system, or have the user typing it in in their application or whatever. You could use it for any kind of runtime flexibility that you need in your system. Okay, that's interesting. So you've been supporting and maintaining ChaiScript for five and a half years.

And I looked at your issue tracker on GitHub, and that seemed very active. Do you have any idea what types of projects are out there using ChaiScript actively? It's kind of funny for you to ask that because I really don't. Okay. It's surprising with an open source project like this, users will ask questions and they'll say thank you when I fix their bugs or answer their questions, but they'll never tell me how they're using it. I know that Open Transactions was one of our first users, and I honestly don't know if they're still using it or not. It's a Bitcoin kind of project of some sort.

And a bunch of game developers, both open source and people who won't tell me what they're working on, have asked about it. And I honestly don't know who all is using it where, but I know that I get enough bug reports and questions that someone's using it. That's interesting. Yeah, I'd really like to know what some of the use cases are out there for it. So what was kind of the reason why you decided to create it? Did you have like a specific use case you were trying to achieve? Well, I guess for a little bit of background,

it kind of started just as an experiment to see if I could write a system that would let me choose at runtime what function to call, like a completely generic, like I was just challenging myself in C++. I wanted to say, I don't know the types of parameters that are going to this function call. I don't know how many parameters are going to this function call. And I don't even know what function it is. I just have a function name and a vector of parameters. Can I make this work?

And I did. And so it developed from there into a scripting language. And I thought the C world has Lua, which is very easy for them to use and integrate into their projects, and it's got a lot of popularity with World of Warcraft, I think was the thing that brought Lua to the world's attention. And I thought, well, that doesn't work very well for C++ because we've got objects and more complicated ways

of dealing with function pointers. Can we make this better? Is it possible to just pass in a function pointer and have the C++ automatically figure out the types of all of the parameters and automatically wrap the function call. And I solved that problem also. And this is something, I mean, it's certainly not new to people who have been programming in C++. There's in Scriptum, which has to solve almost exactly the same problems. And Boost Python, which again, is solving almost exactly the same problems. But, you know, I was challenging myself and went for it and it worked. And then I thought, well, let's see if we can make this just crazy easy to use, not just to the API, but have no dependencies, have no extra libraries or compilation steps or anything to make it work.

And it started out requiring Boost because I needed Boost Function Wrapper, Helper, and I needed Boost PP library. But then when C++11 was starting to get compiler support, I was able to throw away Boost and it requires no external dependencies at all. It's just a header include, and you expose a couple functions, and then you have your easy-to-use scripting language. So it was partially the challenge, and then also I have a lot of experience using Swig, which is an awesome tool. Are you familiar with Swig? A little bit, but please tell me more about it. It's a tool that parses your C++ and generates wrappers for you automatically

and can work with a plethora of languages like Perl, Ruby, Python, two different JavaScript dialects, and others that you can go on and on, C Sharp. I've worked with many of them, but certainly not all of them. So I had a project that was using Swig because I needed to expose Lua and really the functionality that I needed was very small, but Swig was still doing the heavy lifting for me, and I thought, man, it would be great if I could do something that was great for just runtime scripting, so I could get script commands from some external source and just execute them and not need to run the Swig wrapper generator and not need to complicate my build system with extra

steps of having generated code and if i could just throw this in here and bam solve a quick problem and and so that's where we that's where we got to and it's it i know one case that it's being used in where a client actually hired me to do some work is, it's a runtime command kind of thing. It's a system that does dynamic network scanning request. And whenever it gets a command that it's told it needs to scan a new target to see what computer is out there, it can just get the script, and so the script can be dynamic. You never have to update the C++ code running on the end computers. You can just push new scripts to it.

Okay, so that does sound really powerful, to be able to just let your C++ app or whatever just keep running and be able to push in new ChaiScript. Several of the people that have talked to me, again, I don't know if they're actually using the project for this or not, they wanted to do things like scripting of the intelligence in game agents. Okay. So that could save you a lot of development time

if you don't have to recompile your game every time you want to try one little tweak in the AI logic of your whatever game characters. Okay, that makes a lot of sense. So I mentioned that I do a little bit of scripting using a language like Ruby. If I wanted to call into a C++ library from my Ruby script, is that something that could be easily done?

Is that something that Swig would be used for? That's definitely something that you would use Swig for, and certainly not ChaiScript, since ChaiScript is its own language. But Swig, and it really is a great tool if you have a specific need for a specific language, like whatever whatever Ruby. Um, one of my clients is using Swig to, to wrap a gigantic library. I, I'm trying to remember, I, it was something like we have 26,000 functions exposed across the library and that's including, I mean, it's a large object model, so that's all of the overloaded functions of all of the classes in the object model.

They all have to be processed by Swig. Right. Excuse me. So we're using that right now and exposing it to Python, Ruby, JavaScript, and C Sharp and Java. And if that's your kind of need, then Swig really is a great tool for that. Because then, you know, it's adding another language to support. I wouldn't say is trivial for the project like this, but it's certainly not hard.

It's something that takes a few days of work to turn on another Swig generator and add the extra steps to our compilation process and then add some unit testing to make sure that it's doing something sane with the libraries that it generated, but it can do some impressive stuff. Okay. And much more automated than what we can do with ChaiScript. Since we don't have any kind of parser, we have to, you have to tell ChaiScript which functions you want to use, as opposed to just pointing it at a header file and saying, parse this for me.

Right. Back to ChaiScript for a second. You know, one thing I'm wondering is the your C++ code is, I guess, going to call up a ChaiScript function and be able to run it. Can the ChaiScript call back into your C++ and run a C++ function? Yes, absolutely. So I think about the best way to describe this. Everything in ChaiScript is an object, and that includes functions. And you can expose your C++ functions to ChaiScript, or you can create a function in ChaiScript and pass that back. So you actually have like a, any, you know, whatever dynamic function you want to think of.

I don't know. It's hard to describe over the audio, I guess. But if you have a function foo, and it takes three parameters, so it's ChaiScript is, um, strongly typed, but it's dynamically typed. I think I've got that right. So an object, once it has a type, is strongly typed and it keeps that type. You cannot ever change the type of an object. Sure.

So if you pass, you create your ChaiScript function that takes three parameters and all three of them are untyped. You can do whatever you want to do with them. You can then pass, actually pass that function back to your C++ land and have it as an std function wrapper. And call it as if it were a strongly typed C++ function from C++ land. Or use it from ChaiScript as a dynamically typed function in ChaiScript land. I've tried to make every aspect of this orthogonal. If you throw an exception in ChaiScript, then that exception gets popped back up to C++, and you can catch it there. Or if the C++ function that you're calling from inside of

ChaiScript were to throw an exception, you can catch that inside of ChaiScript if you want to. Oh, wow. Okay. Very interesting. Another question, when you're first writing your ChaiScript functions, is there any way you could test those like on a command line? Or do you need to load them up from a C++ application in order to run them? We have a simple command line tool that you can test, you can play with and use the built-in functionality, but you'll definitely have to integrate it with your own application if your point is to call your C++ functions, testing calling those.

If you just want to play with the language, absolutely. There's chai.exe or whatever, depending on your platform. Right now, my automated build environment builds for Visual Studio 12 and 14, 64-bit and 32-bit, and Clang on macOS, Clang on Linux, and GCC on Linux. Okay, very interesting. So this is definitely going to be something that I'm going to check out. So another project that you started recently, which I mentioned in your bio, is cppbestpractices.com.

What's your goal with this project? I just kind of wanted to write down stuff that I thought that I've learned over the last 12 years of programming in C++. I started to wonder if I was starting to forget things at this point. And then I thought, well, maybe this could be interesting to other people. Maybe it could fill a gap. I'm not trying to get down into the nitty-gritty details like Myers does with his C++ books.

I just kind of want to cover the bigger picture. It's a good idea to have a continuous integration build environment so every commit that you make, you can see the results of it as quickly as possible. I've made probably some controversial statements about what I think is a good idea for indentation and spacing and that kind of thing. But just stuff that seems to have worked well for the development I've done. Yeah, those that kind of thing. But, you know, just stuff that seems to have worked well for the development I've done. Yeah, those can be religious discussions. They can be, absolutely.

When I make a suggestion about how many spaces you use, I do give an argument for why, but, you know, for that particular part, I say, you know, what matters is that you're consistent and the entire code base is the same. Right, right. Are you hoping to get contributors adding to this project? Absolutely.

It's on GitHub as an open source book of sorts. I was actually just looking at gitbook.com this morning to see what the possibility is of having them like automatically generate PDFs and stuff, but I didn't get very far with that yet. Uh, but it's, it's on GitHub. Anyone could fork it and make their own modifications, create pull requests. You can use it as the basis for your own, um, internal coding standards document. If you want to just fork it, make whatever changes are, are reasonable for your organization. Just, I don't know, kind of made sense to have a place for the community

to collaborate on what are the best practices. We'll see if it goes anywhere. Well, it sounds like a great idea. I'm definitely going to check it out. So, is there anything else you want to talk about before I let you go today? I think that pretty much covers everything that I've been looking on lately. Okay. Well, your bio listed you as an independent contractor. How can someone go about hiring you to get advice about cross you know, cross-platform C++ development,

which seems to be your specialty? Well, I guess my email would be the best. That's jason at emptycrate.com, which is also emptycrate.com is my blog site, although it's been a little neglected for the past couple of years well i saw those you had some recent articles there you had the ife article right yeah it's just i used to make more of a habit of it i would blog about whatever c++ book that i had been reading recently there's really old posts there that i i blogged like as i was reading um the C++ in a nutshell, which is quite a tome of little details to have written down. But I know it's just not as much writing

as I would like to be doing necessarily. Yeah, I know I would always like to be writing more too. There's only so many hours in the day though. Yes. Well, thank you so much for coming on the show. And where else can anyone else find you online i guess emptycrate.com are you on twitter i am on twitter it's uh i am lefticus which is l-e-f-t-i-c-u-s i don't tweet a lot uh but if you send me a message there, I'll get it. Um, lefticus on GitHub also, um, which is really where most of my activity is this day. I've had the great fortune. I would say

truly, uh, the best part of my career for the past five years is almost everything that I've been paid to work on has been open source. Oh, that's amazing. Yes, it's great, and I would love if that can keep going, but it seems like I can't imagine that lasting forever, but it has been a great opportunity. And if someone wants to find this CPP Best Practices, that would be on your GitHub, correct? It is, or you can go to cppbestpractices.com, which redirects to the GitHub site right now.

Okay. Thank you so much, Jason. Thank you. Thanks so much for listening as we chat about C++. I'd love to hear what you think of the podcast. Please let me know if we're discussing the stuff you're interested in,

or if you have a suggestion for a topic. I'd love to hear that also. You can email all your thoughts to feedback at cppcast.com. I'd also appreciate if you have a suggestion for a topic, I'd love to hear that also. You can email all your thoughts to feedback at cppcast.com. I'd also appreciate it if you can follow CppCast on Twitter and like CppCast on Facebook. And of course, you can find all that info and the show notes on the podcast website at cppcast.com.

Theme music for this episode is provided by podcastthemes.com.