CppCast - Exercism.io and Refactoring

Episode Date: June 24, 2015

Rob and Jason are joined by Richard Thomson to discuss exercism.io and C++ refactoring tools. Richard Thomson is a passionate software craftsman.  He has been writing C programs since 1980, C++ programs since 1993 and practicing test-driven development since 2006.  For 10 years, Richard was a Microsoft MVP for Direct3D, Microsoft's native C++ API for 3D graphics. His book on Direct3D is available as a free download. Prior to that, Richard was a technical reviewer of the OpenGL 1.0 specification. He is the director of the Computer Graphics Museum in Salt Lake City, Utah and currently works at DAZ 3D writing 3D modeling software in C++. Recently, Richard has added the C++ language track to exercism.io and has been working on adding refactoring tools to the clang tool suite. News C++11/14/17 Features In VS 2015 RTM Futures for C++11 at Facebook A Conclusion to Accelerating Your Build with Clang Live Webinar: A Tour of Modern C++ Richard Thomson @legalizeadulthd Richard Thomson's blog Richard Thomson's Github Links Utah C++ Users Group Create your own Refactoring Tool in Clang CppCon 2014: Matt Hargett "Common-sense acceleration of your MLOC build"

This episode of CppCast is sponsored by JetBrains, maker of excellent C++ developer tools. Listen in for a special discount code later this episode. And by CppCon, the annual week-long face-to-face gathering for the entire C++ community. Get your ticket now during early bird registration until July 10th. Episode 17 of CppCast with guest Richard Thompson, recorded June 24th, 2015.

In this episode, we talk about accelerating your build with Clang. Then we'll interview Richard Thompson about Exorcism.io. We'll also discuss the latest refactoring tools and the benefits of automated refactoring. Welcome to episode 17 of CppCast, the only podcast for C++ developers by C++ developers. I'm your host, Rob Irving, joined by my co-host, Jason Turner. Jason, how are you doing tonight? I'm doing all right, Rob. How are you doing? Doing good.

Any news with you? I know we're still both waiting on our CppCon submission announcements. Yeah, that's the main thing I have going on right now. Yeah, they sent out an email to everyone who submitted recently that I guess they got overwhelmed by the number of submissions this year, and it's taking them a few more weeks to send out notifications to everyone who's getting their talks accepted. That's pretty crazy. I'm pretty sure last year they actually had to extend the submission deadline because they didn't get enough submissions. I might be thinking of C++ now, but I think CppCon might have had that problem too. Okay, well that's definitely very exciting that they have so much interest in speakers there. Very cool.

Okay, so at the top of every episode, I'd like to read a piece of feedback. This week I got an email from Henrique. He writes in, well, first of all, thank you for the podcast. My background is graphic and motion design, but I've been coding for some years now doing some interactive installation work, especially with Cinder and C++, and it's nice to have a place to just listen to some general topics. Anyway, I was surprised to discover that the header and footer of your page is actually Cinder code, so I think it's appropriate to ask for a podcast with Andrew Bell, one of the creators of Cinder.

He gave a talk at the I.O. Festival back in 2012 that really changed the way I see the creative coding field, and I'm sure he'll have a lot to add to the podcast. So I was not familiar with Andrew Bell, but I would definitely like to talk to someone from the Cinder community, especially one of the creators. So we'll have to reach out to him to see if he'd be interested in being on the podcast. And our guest tonight is actually the one who suggested I use Cinder for the background image code. So let me go ahead and introduce him. Richard Thompson is a passionate software craftsman. He's been writing

C programs since 1980, C++ programs since 1993, and practicing test-driven development since 2006. For 10 years, Richard was a Microsoft MVP for Direct3D, Microsoft's native C++ API for 3D graphics. His book on Direct3D is available

as a free download. Prior to that, Richard was a technical reviewer of the OpenGL 1.0 specification. He is the director of the Computer Graphics Museum in Salt Lake City, Utah, and currently works at DAS3D, writing 3D modeling software in C++. Recently, Richard has added

the C++ language track to Exorcism.io, and has been working on adding refactoring tools to the Clang tool suite. Richard, welcome to the show. Thanks for having me. Thank you for being here, and thank you again for pointing out the image in the podcast website

that was using some C++ CX. I finally went ahead and changed that a few weeks ago. Cinder is a great example to use there, and it's a really fun library to play with. That was one of the most heavily attended talks that I've given at our Utah C++ user group. Very cool, very cool. Yeah, we haven't talked about it too much.

We did do another talk a while back with Vittorio Romeo about kind of simple game libraries in C++. And I think we talked about two other ones, but I don't think Cinder was one of the ones he had too much familiarity with. I think it was SFML was the main one we talked about. That's right. Yeah. Yeah, I've got to, before we move past your bio, I've got to ask about this Computer Graphics Museum that you are a director of, Richard.

Well, it's got a website, computergraphicsmuseum.org, and it primarily consists so far of a collection of hardware artifacts, and I have it stored in a warehouse in Salt Lake City. I moved to Salt Lake City to study computer graphics at the beginning of my career here and ended up working at graphics companies off and on and um there's a lot of history associated with utah through the universities of the university of utah and brigham young university a lot of core graphics technology was created and came out of those institutions um yeah the uh if you want to get a real kind of visionary experience of the beginnings of computer graphics, I believe you can watch it on YouTube. Watch Ivan Sutherland's Sketchpad demo.

This is him demonstrating the Sketchpad software that he wrote at MIT in 1963, the beginnings of interactive computer graphics. And you look at what he's doing, and it's very much the same things we do today. And now we call them content creation applications, modelers. Photoshop falls into that category as well, but that's more image manipulation than modeling. And you can see that what he did in his PhD was the beginning of this entire field. And although he didn't create it from whole cloth, there were some things that had gone on in the 50s, but they to late 60s, to come to University of Utah and start a concerted emphasis on studying computer graphics because the University of Utah Computer Science Department wanted to establish itself as preeminent in some specialty of computer science. And they thought computer graphics was a good one. So they recruited Ivanivan sutherland and the department head was david evans and then as they worked on their research

and brought in graduate students to do more research um they eventually formed a one of the first commercial companies for computer graphics which was called evans and sutherland um they ended up primarily focusing a lot on military and commercial flight simulations. So they're not so much of a household name. But Evans & Sutherland invents the frame buffer. Back when a frame buffer cost like half a million dollars. A dollar per bit.

Memory was really expensive. Right, right. And a lot of the core research that was done by graduate students at the University of Utah, like Henri Garot, who does his research, his PhD thesis on Garot shading. That is used in every single 3D graphics game that you play. That's got Garot shading going on it. James Blinn does his work on texture mapping and bump mapping and that is also a technology that's used in almost every modern game he goes on to do a lot of that animation work at nasa jpl for the animations

showing the flybys of those satellites that are exploring the outer ranges of the solar system in the mid-70s. In the late 70s, he's doing that. You've got Bui Tui Fong who describes Fong shading, which is also used in modern graphics. And there's a whole lot more. I'm just mentioning some of the names that people

might recognize. The founders of Pixar come out of the University of Utah. James Clark, who founds SGI, come out of the university of utah james clark who founds sgi comes out of the university of utah wow um uh david warnock who founds adobe comes out of the university of utah so a lot of the key players in the giant uh that formed the giants of computer graphics they all have not all but many of them have a strong connection to the university of utah so and there's still lots of things going on today it's just um that so many people graphics has broadened into such a huge field that now so many people are contributing to graphics that it's hard for any one institution to dominate as much

as university of utah did in the early days but the there's still graphics research going on at the university of utah there's uh interesting things going on with scientific visualization medical applications all kinds of things just now graphics has become so diversified like any other field of computer science that uh you know it's difficult to um you can still contribute but it's difficult to make pioneering contributions in a brand new area because graphics has now been a field that's been going for 30 plus years and it's pretty mature but that's not to say there isn't interesting new things going on. So what kind of artifacts do you have in your startup museum and what kind are you looking to acquire? So I've concentrated in acquiring artifacts that are the pre-PC era,

so the pre-microcomputer era. So that's workstations, many computer-oriented systems, and if you on the website, there is a link at the top called The Collection, where you can see some overview pictures of all the hardware that was

in the collection at the time I took those pictures. Okay. And my hope is to turn the collection and its artifacts into exhibits where you can experience the kind of technology either through some kind of replica or through preferably actual equipment. So I try to focus on getting multiple instances of actual equipment so that you can have a hands-on actual experience for the computing,

for the era of computer graphics that that exhibit would represent. But that's the idea, is to make it an interactive museum. But at this point, it is, if you visit, it's a static collection, but I'm happy to interpret if anybody comes and wants to get a little tour. It's not open to the public at this time because it's basically just me, but there's a plan.

And that's the vision. the excellent science museum called the Exploratorium in San Francisco. That is kind of my model of hands-on interact, explore, experience. And with graphics, you have the additional avenue that you can act, interact with the exhibit as your own content creator. Maybe you mash up some other content that's already part of the exhibit, create a new thing, a new model, a new image, a new something. And then you leave that for the next person who can then interact with the equipment further and mash up what you left behind and move it forward. And then of course, in the modern world, all that, everything that you make by interacting with the exhibit could be shared through social media. So those are kind of my ideas for participation and how to make a museum that's fun and interesting, as opposed to walking through walls and walls of corridors and corridors of glass cabinets with little printed signs behind the glass.

And you're like, okay, I'm kind of tired reading now. Well, it sounds very interesting it does yes if i ever make my way out to utah i hope the museum's open by then uh if you visit i'm you know just give me a call and i'd be happy to give you a little personal tour okay great well i know that i can extend that invitation to everybody that might be listening to this podcast but you but I'll do it until it becomes overwhelming. Great. Well, I wanted to talk about a couple items of news before we get to talking to you, Richard. The first one is coming from the Visual C++ blog, and this is about C++ 11, 14, and 17 features coming in VS 2015 RTM. It feels like whenever we have a news item from the Visual Studio blog,

it's usually a big table, and this is another one. And it's basically going over some of the C++ 11 and 14 features that they've built into the standard library, which will get released with VS 2015. So great work from the Visual Studio Plus team catching up with Clang and GCC in the standard library implementation. Do you guys have anything to add to this one? There's a couple of things on here that make it look like they're planning another release

of Visual studio 2013 another update to it it's kind of implied yeah it feels like it's kind of implied because this make unique currently in 2013 is alias templates for type traits are those currently in 2013 i didn't think they were but i didn't go back and verify well this, this is all... Oh, okay, right. Those last few features are 2013. Yeah. Yeah, I guess maybe they'll do... What is it? Update 5 or 6, would that be, for 2013?

I think it's 5, maybe 6. I don't know yet. Yeah, I think there's at least four updates out now. Yeah, I know that, because some of the software I work on requires update 4 to not crash. Right. Yeah, I guess they're planning on doing another update

for 2013. They must be. Okay. You know, you could ask Steven... Let's see if I not butcher his last name. It's hard for me to pronounce. Leviev? I don't think that's right only because

I don't think that's right either. I just say STL. Just STL. Yeah, that's what he says to call him. So I'm pretty sure if you commented here... Let's see, got the comments open. Yeah, if you commented... I haven't read through the comments.

But if you comment and asked on that, if it's not already asked, I'm pretty sure he would answer. He's very responsive. He likes to put these tables together because it's the never-ending discussion thread of what does, you know, people say, well, Visual Studio, it's not C++11.

Or C99 yet. It's like, yes yes well that's true um it may be c++ 11 enough for you and the only way to know that is to consult this table and see if the things that you use are properly supported um i know uh the boost guys and uh maybe jason's in this camp too because he's a library implementer they really struggle with the pre-processor i don't have a problem with the pre-processor personally i try to stay out of pre-processor land um as much as possible the uh a lot of the boost guys are you know doing those kind of crazy tricks and they they struggle to uh you know variatic macro support and things like that there's certain things that give them grief and only on

msvc and it's only because of the pre-processor um i read between the lines from what steven's saying that they're c90 you know that they're working on c99 pre-processor support and they're saying like it's kind of there but it's a little buggy yeah that tells me that they're working on C99 preprocessor support and they're saying it's kind of there but it's a little buggy that tells me that they're listening to that pain and trying to do something about it so that's encouraging but it's one thing

to say well it's not C++11 or C++14 but to know whether that's really useful for your code base you kind of have to consult this table and be like okay the things it doesn't have are stuff I don't use so it's good enough for me and it's tougher as a library implementer because those edge features that they haven't implemented yet

at the edge of their roadmap I mean as a library developer that could be central to what you're trying to do with your library. Right, exactly. It's a mixed bag. This next article is coming from Facebook. We actually talked about Facebook last week, too. This one is about a new asynchronous library that they put out called Folly Futures.

And it's a pretty long blog post. I wasn't really sure what the benefit was of using Folly over something else like HPX. And I saw some of the Reddit commenters were saying kind of the same thing. It also seems very similar to the Futures library that Sean Perrin was talking about with us. I'm not familiar with HPX. It sounds familiar.

I might have looked at it before, but off the top of my head, nothing's coming up. But when I read this, I was like, to me, I read this, I'm'm like this really feels like the async library in javascript land which if you've done any node.js programming where everything is asynchronous it's like that async library just ends up being the first thing you end up using everywhere that you're doing any kind of IO, just because it's the only thing that makes your code understandable.

And it has very much the flavor of that Facebook Futures API, has very much a flavor of async and promises in JavaScript. There's a, I believe it's called Q Promise Library for JavaScript. There's like a miniature version of that that they use in Angular, AngularJS, for their promises. And it's a bit of a mind bender to get into this asynchronous stuff because you're used to writing kind of sequentially. Like you just read from top to bottom and that's what happens. And then you start dealing with all these callbacks and it gets kind of, you know, a callback inside a callback inside a callback and your brain starts to melt down.

I like these futures and promises libraries that help you kind of re-linearize that so that when you read your code, it feels sane again. Because code is going to be read a hundred more times than it's going to be written that's a good point if anyone's interested there's also an article from instagram's engineering blog about the same future library and if i understand correctly um instagram is owned by facebook so yeah i believe it kind of makes sense i'd imagine their engineering teams have moved pretty close together working on these in conjunction um jason did you want to introduce this next article sure uh so the next article is

a uh it's a conclusion to accelerating your build with clang That's the second of a two-part series of articles on various methods that you can use to speed up your build. It introduces some good topics, like using Ninja as a target for CMake instead of using Make. And just in general, using clang is faster than gcc and using the gnu gold linker can help and then a couple of things that stuck out to me here that i'd never heard of before like split dwarf debug files yeah i never heard of that as well it's like apparently saves your debug information in a separate file

from the binary. So it can speed up link time since the files that it's trying to link are smaller. And I think you told me that you were working on implementing some of this with your own code this morning? Well, I was reading the Reddit discussion

about this, and in the Reddit discussion they mention an add-on for CMake that isn't really related to this article at all, but it's an add-on for CMake that's supposed to make your builds faster by doing things like merging all of your C++ files into one file, which I had never heard of before and terrifies me. It's the so-called Unity build. Yeah. Yes. The Unity build. Thank you. I couldn't remember the name of it. It says right in the description that this breaks C++ and you'll probably have to modify your code to get it working. So I said, okay, moving on. And it also is supposed to give you a painless method of doing pre-compiled headers, which is really my interest in it. And unfortunately, I don't remember what the name of that add-on for CMake was. If anyone else knows, otherwise, I'll try to look it up real quick. I think I've stumbled across it before,

but I can't remember the name for it. But if you search for CMake Unity Build, I'm sure it will come up. Yeah, I'll get there in a second. But otherwise, did either of you guys take the chance to look at this? I read through it, and because I'm currently a Clang contributor, I'm going to go and try out some of these things. Oh, okay. I'm not contributing to the compiler.

I'll just clarify that. I'm not contributing to the compiler. I'll just clarify that. I'm contributing refactoring tools. I'm going to try some of this stuff. I was already trying... Some of this advice is similar to a talk from CppCon

2014, how to optimize your million lines of code build. That is a good talk yes and uh there was some stuff in there that i was like oh why didn't i think of that and now that he's told it to me it feels so obvious and i feel so stupid for not thinking of it before um and a lot of the things that he was talking about were similar uh the you know build yourself your own compiler that is optimized for compiling your project which sounds super crazy yeah you know it's just kind of like whoa mind blown kind of thing but when you think about it you're like wow that actually

is a really good idea um something that's an option for people that use an open source compiler i we primarily where i work use uh visual studio so that's not so much of an option for us but um you know it's a really interesting idea and um it then gave me the idea of like optimizing my own application we use qt we use a build of qt that we compile we should program our profile guided optimize qt for our application right so the same idea just i'm not optimizing the compiler i'm optimizing my own app and that's just kind of like for people who are old hands at optimization that might be just kind of like well duh who are old hands at optimization, that might be just kind of like, well, duh, yeah, but I haven't thought of it. So what kind of results did you get by making a PGO build of Qt? This is on our list of things to try right now.

Oh, okay. We just finished a big release to showcase some new NVIDIA GPU ray tracing technology, our current was our crunch that we were working on so it might bubble up to the top soon um and then a trick that i did start using with my clang development was put your whole source tree in a ram disk kind of again duh but good idea um i'm gonna try some of these things because he is writing that blog post from the point of view of a developer contributing to clang yes and he says you know you do a clean build it might take you a few minutes i'm like if only it were a few minutes it's like 20 for me anyway um it's not a particularly old machine that I have that I'm developing on, but I built it on my laptop, and it ground away for a long time with all the CPUs at 100%.

And then I just looked at the size of the tree that resulted, and it was like 10 gigabytes of compiled code. I mean, it's not small. So I'm going to go and try a bunch of these things to see how much of a difference it makes for me. But I definitely recommend if this blog post was interesting to you, I definitely recommend going and looking at that, how to optimize

your million lines of code talk from CppCon 2014. It's on YouTube and on Channel 9. I don't think I watched that particular CppCon talk, but I'm going to have to look for it, and I'll add a link to the show notes. And I did find that

CMake module. It's called Cotire. C-O-Tire. And it can add precompiled headers and Unity builds to your CMake projects. Very cool cool unity builds are pretty you know flame inducing conversation thread in this community uh there are people who swear by them and then there are people who you know warn you off of them very strenuously i haven't ever

tried it um intentionally i did work at a company whose development practices were just so antiquated that that was how they did their build because they didn't understand make files. They just took all their.c files, concatenated them into one big file, and then compiled it. I will save them embarrassment by not mentioning their name or the product. That sounds like a good plan.

Okay, so the last thing I wanted to mention here was JetBrains has announced that there's going to be a live webinar, a tour of modern C++. And I have this link here, which is actually to register for it. And they opened up a second session, which is more for North American viewers.

It's going to be July 2nd, which is a Thursday, from 2 to 3 p.m. And Dimitri Nustruk, who is a developer evangelist for JetBrains, is going to be doing the presentation on Modern C++. So definitely should be interesting to see what he has to say and go over, and I'm sure it will be a good session. Was that 2 p.m. Eastern? Yes, 2 p.m. Eastern. Okay, so during your bio, Richard, you mentioned exorcism.io.

I did a little bit of research into exorcism before reaching out to you, but could you tell us or tell listeners a little bit about that project? Sure. This is a project created by Katrina Owen, who was doing some mentoring workshops with other developers, focusing on test-driven development or acquiring a new language, learning Python or learning JavaScript, learning Ruby, whatnot. And she noticed that, and I'm paraphrasing, so I might get the story, some minor details wrong, but I think I got the gist of it. She noticed that a lot of the progress in people's ability to understand exercise, but it came when they took their different solutions to an exercise and got together and discussed the different ways that people had solved that exercise. And they got feedback from their peers on like, oh, I see you did this, but in JavaScript you can do this sort of thing, and the code is shorter and it's more idiomatic it's more like what a javascript programmer would be used to seeing and when i read about that i was like you know what that is something the c++ community could really benefit from because there are things that are idiomatic in C++ and Bjorn Straustrup explains them to you in his book. And he says, this is the way I recommend you do things and not the C style way.

But then when you go out on the wild internet and you start looking for code examples, it's like, I come across 99 crappy examples for every good example that I find, right? It's all full of strcpy, strcat, strlower, all problematic and error-prone C-style APIs and no use of RAII. We really need a better name for that. No use of smart resource classes to create intrinsically exception safe code no use of standard containers no use of standard string um just save yourself all that misery and grief of coding your own container right just use the standard containers unless you have actually measured that it matters to use your own container and so you know so many times I've tried to give this advice at various places in user communities and places where I worked.

And my experience at my workplace was always, I would give this training, say, here's how to use C++ like C++ and not like C. And the end result was entire categories of bugs would disappear. Okay. We just didn't make those mistakes anymore. And I thought, you know, if exorcism is a place where you can learn that without me having to be in the room,

right? I can only communicate to so many people at a time in person. But if we can collaborate kind of, you know, crowdsourcing style, then maybe this wisdom will reach people faster. And they did not have a C++ language track. I experimented with their JavaScript track and the Python track. And my experience was I solved the first program problem in JavaScript. And although my solution passed the tests, I knew it was pretty ugly.

But I just put it up anyway. And once you put up the solution to an exercise, you can go see what other people did. And so I looked at, even without waiting for somebody else to tell me how much my solution was ugly, because I knew that it was ugly. I'm like, all right, now that I can see how other people have done it, what can I learn from just looking at their code? And within five minutes, I was like, oh, that is so much better than what i did quick let me update my version of solution to steal their idea and so i'm not embarrassed anymore and i had a similar experience in python and i have been programming even a shorter period of time in

python than javascript i've javascript doing off and on for a while since, you know, probably 2000 or something. Python only a little bit over the tests, but we need a reference solution that embraces modern C++. And so that's what I strove to do was stick with a standard library. If there's a hole in the standard library, go for boost. Try to do things in an idiomatic C++11 style way. And it was a lot of fun to code those solutions, those exercises, and make them available. And there were a few things that were a struggle, which is how to get this to work in a reasonable fashion

across Mac and Windows and Linux without making any one group complain too much. There's always going to be some complaining, just because it's hard to make everyone happy. But so far, it's worked out pretty well.

I use CMake for the build system so that I don't have to maintain multiple projects for the different platforms. I use Boost Test for the test framework. The test framework itself, the choice is rather arbitrary, but

there's so many other things in Boost that you're going to want for more advanced problems, like date computation and calendar computation and stuff like that. It's not anything good in the standard library yet. So we want like boost date time, things like that to do Gregorian calendar date manipulations, et cetera.

So I chose boost to include as part of that. And it's been pretty well received. It gotten a lot of participation, more participation than it's possible for me to go and comment on everybody's solution. But what I did was take all my reference solutions and made them available as, quote unquote, my solution so that other people, once they solve any problem, they can at least see my example solution. So they can't just be like, well, I'm the first one. I finished it. And now there's nothing else for me to look at.

So at least there's one for you to look at. And that was a really good idea because I got, even though I thought my solutions were pretty good for C++, I still got useful feedback on like, oh, notice you're doing this, but Boost already has a function that does that, and you can kind of squish those five lines in your own function down into just one line. I'm like, okay, that's good.

So it's been a lot of fun. A lot of the exercises originate from the Ruby community, the JavaScript community, the Python community. And I've kind of followed the Ruby track in terms of the sequence of the exercises. And I kind of imitate JavaScript track for like the tests that might be interesting to write. But my hope is to extend this to the point where we have exercises that are unique to the language characteristics of C++. How can you come up with a sequence of exercises to introduce people to the idea of generic programming,

of template metaprogramming, not to the extent where I'm expecting them to go and implement a full kind of complicated expression template library or something, but enough that these things aren't scaring them anymore. You know, that they understand it enough that they realize what's going on, how the machinery works. And so that I think is interesting because for generic programming and this kind of template metaprogramming stuff, really it's C++.

There's not very many other languages out there that will let you explore that. It's kind of advanced stuff, and it's scary to a lot of people, but I think it can be demystified and made more approachable with a sequence of exercises designed to illuminate rather than confuse. Right. I did look at one of the examples that comes from exorcism, which I guess is one of the examples you coded up, and it looks like you basically get a pretty simple starting point with a lot of different tests. So is exorcism meant to be a way of learning test-driven development if you're not familiar with it?

So that's an interesting question. Kind of implicitly built into the culture was an assumption that you know test-driven development. Okay. And what I did in my kind of orientation documentation to the language track was say, if you've never heard of this before or you've never done it, go look at the material that i created for the workshop for c++ now 2014 because that has uh slides uh power you know pdf powerpoint style presentation that explains everything every step of the way and also shows i have matching code so that you can get the code that is associated with every slide so you can

you literally can just follow along step by step and do it yourself without having to know how to do it per se and so exorcism kind of does half of the tdd for you in that every exercise comes with a set of tests, and the tests are already written. So you're not really doing test-driven development in the sense that you're writing a test and then writing code to make the test pass, because you have the test, so you write code to make the first test pass, and then you write more code to make the second test pass,

but the tests are all written for you now what's kind of interesting about this is um for c++ you know i write the test for you but i'm not supplying the syntactical boilerplate that's necessary just to even get the test to compile so you have to really look at that test and go, this test is expressing something, some behavior about a piece of C++ code. What is the structure of the C++ code that will even syntactically satisfy what's being expressed in the test? And then if I can get it to compile that errors, is it going to satisfy the semantics of what is in the test, the behavior that's expected from the test,

so that the test will then pass. And it's kind of an interesting way to experiment with TDD because I find a lot of beginners to TDD where they struggle is, what is the first test I should write? And after I get that test passing, what is the next test i should write and after i get that test passing what is the next test i should write so those barriers are gone because i've written all the tests for you um and it's interesting to see how people interpret what is this test intending me to write um and i think that leads to and the point of exorcism is not to

quote unquote solve the problem the the solving the problem is just a mcguffin to get you to the point where we can have a conversation about your solution and so that's why you know all this reference solutions around github you can go and browse them and you just be like, aha, look, I solved your problem by just stealing your code. I'm like, great. Now let's have a conversation about the code. Because the point is that conversation, that that's where we gain more insight about how to write idiomatic C++, modern C++, how to write C++ that is robust in the face of exceptions, how to write C++ that is revealing its intent by simply reading it. You know, not a shotgun blast, a punctuation to the face that I can't understand. It's something that I just read it and it's like, oh yeah, I know what this is doing

because I just read it and that's like oh yeah i know what this is doing because it just i just read it and that's that's all i needed to do um i was applying for a job once and they gave me a programming problem and i did the programming problem and i did it with test driven development i give them the code and they're like wow that's amazing it's first time anybody ever wrote unit tests for one of these problems and i was like really and they're like yeah sometimes we get code that doesn't even compile i'm like well i hope you didn't hire those? And they're like, yeah, sometimes we get code that doesn't even compile. And I'm like, well, I hope you didn't hire those people. And they're like, no, no, no, no. And then I was like, well, do you have any questions about the code I wrote? And they said, well, I noticed it doesn't have very many comments in it. And I was like, yeah, did you have

a hard time understanding the code? And they're like, no. I'm i'm like well that's why it doesn't have any comments because they would have been superfluous right so do you think um it sounds like exorcism is a great tool to use no matter what your skill level is to just become a better programmer would you agree with that i i would agree with that um it's not a good way to learn c++ if sure you know i would not and i tried to uh explain that in the orientation material the documentation that i wrote up i was like if you haven't learned c++ yet you might want to try some of these other places that are designed to teach you c++ first and And I did some Googling and found some websites that seemed to be semi-respectable.

There's an approach to learning a language that I think is interesting called programming coens. Are you guys familiar with this? Coens? No, I don't think so. It's an approach to teaching a language where they just kind of show you some code, and the code has a placeholder in it where you're supposed to replace that placeholder with some kind of syntactical construct.

Programming Mad Libs? It's kind of like that. Except each one is designed to kind of incrementally build on the previous one to introduce like, oh, well, this is how I write constants. This is how I write expressions. This is how I do comparisons. This is how I make a function. This is how I make a class. And there's some people that have tried doing that for C++, but I wasn't, you know, I wasn't satisfied with the results, so I didn't send anybody to one of these Cohen attempts.

It's spelled K-O-A-N. I've done some exercises at the Utah Software Craftsman meeting where we've tried to learn a new language by, as a group, we're just going to work on these coens and make them all pass because they're all expressed in the form of a test and um we did it for closure which is a kind of functionally oriented lisp dialect for the jvm it's kind of fun it doesn't suffer from from incredible amounts of parentheses like standard Lisp. And that was kind of fun. It was a fun way to get an introduction to what does Clojure feel like to write programs in it. We're not solving a particular problem. You're just kind of experimenting with the constructs of the language to get familiarity with them. And I think that could be

good for C++, particularly since now we have that IDE1 website where you can just kind of type some C++ in and get it compiling and running and see the output without having to, you know, download five gigabytes of compiler and all that stuff. Yeah, I really like those websites as tools, just being able to type something in really quick and see what the results are.

So, yeah, I can see using that. I want to wrap this discussion for just a minute to bring you a word from our sponsor, JetBrains. C and C++ have a long history, going back to the early days of programming itself. Still, it's hard to find a good development tool for these languages. Luckily, our good friends at JetBrains, after spending over a decade making all sorts of tools for a great many technologies, now provide C and C++ developers with three dedicated tools, CLion, ReSharper C++, and AppCode. All three take care of the routine and help developers focus on important tasks.

They natively support C and C++, including C++11, libc++, and Boost. C++ templates and macros are resolved correctly and supported throughout each tool. Find your way through the code quickly with hierarchical views and instant navigation to a symbols declaration. Boost your productivity by generating the missing members with override implement actions. Rely on code refactorings and be sure that your changes are applied safely throughout the whole code base. Write better, safer, and more efficient code with on-the-fly code analysis protecting you from errors and redundancies before you even compile. Choose one of these three tools depending on your needs. For a cross-platform C++ IDE, choose CLion. If you work in Visual Studio, go through Sharper C++. Or if you develop for iOS and OSX, use app code.

Visit jetbrains.com slash cppcast dash cpp to learn more and download your free evaluation. Or get a private license at a 25% discount using the coupon code cppcast jetbrains cpptool. And if you're a student or an open source project, use all of them for free, courtesy of JetBrains CPP tool. And if you're a student or an open source project, use all of them for free, courtesy of JetBrains. Well, let's move on to talking about refactoring. You gave a really good talk at C++ Now last year. And one of the first things you go over in the talk is talking about the lack of refactoring tools for C++.

And I wanted to ask, do you think there's been much improvement over the past year since your talk? Tremendous improvement. And I attribute that to several things. And they're all tied to the releases of tools. So CLion has been released. And that has, as I would expect from being a product of JetBrains.

JetBrains, by the way, is just they're the king of refactoring in Java land. Just like nobody does it better than them. They just blow everybody else out of the water. And as somebody who's really keen on refactoring, I would do refactorings on Java code in a JetBrains product. And the product is just saying to me like, hey, I noticed you just did that. Do you want me to do that over here too? And it's just like, oh my God, you're awesome. Right? I mean, that's what I want from a tool. That's truly an assistant, right? It's just like, hey, I noticed you just extracted that method.

It consists of these sequence of tokens. You got the same sequence of tokens over here five source files away wow uh do you want me to just fix that too oh and by the way in that other one they it's like they named all the variables differently but it's the same thing do you want me to just fix that for you you're like yes please oh my god that's so awesome so do you find that sea lion works that well also uh sea lion is a 1.0 so we you know we shouldn't expect it to win the gold medal you know on its first year of living but it is doing very very well now i've been playing with c++ refactoring tools for some time now and the initial ones that I tried were so disappointing that I actually created a test suite

to judge how good a refactoring tool is. Interesting. And that test suite is out on GitHub, and it's unfortunately not very automated, so it's kind of exhausting to run the tests because you have to manually do them because it's it there's a lot of questions about how could you automate it and there's so many variables it's difficult but um i ran that test suite against an early access preview

release build of sea lion and it did better than any other refactoring tool i've ever run that test against so their rename refactoring is very high quality it gets very few things wrong and there are a lot of ways in c++ that names can interact in such a way that it is easy for a tool to get them wrong yeah um it's not perfect but it does have a very high score. Now, they also have a Visual Studio add-on called ReSharper for C++.

And I've used ReSharper for years on C Sharp code, and it is just every bit as good as IntelliJ at refactoring and making suggestions, doing quick code fixes, and doing kinds of static analysis on your code and putting little red squigglies under things that are suspect. And that is very much the experience in IntelliJ.

That is also the experience in CLion, doing that kind of background static analysis on your code and highlighting things that it thinks are suspect. I primarily use sea lion in fact i exclusively use sea lion when i'm contributing making contributions for clang and the clang code base is you know pretty advanced code it's large it's got lots of kind of code generated stuff slurped in through macros and include files and things. So CLion still puts red squigglies under some things there that are not accurate in the sense that there's nothing wrong with the code. There's no reason to highlight a problem there. The problem

is the code is complex enough that it's still confusing CLion. But I've also used CLion on one of my own open source projects that I'm trying to modernize, and there it does really well. It's a smaller code base. The code constructs are not quite as crazy. And in all fairness, when I say crazy, I mean it's just kind of more use of the preprocessor. Clang makes heavy use of the preprocessor. Clang makes heavy use of the preprocessor

and code generation to eliminate boilerplate and to have a single source of knowledge for the description of something that's involved in a compiler. There's lots of, as you might imagine, for different backends, lots of things that are generated from a description of the backend,

and the description is not in C++. It's in some kind of domain-specific language file that gets parsed and anyway. Uh, so, uh, but C-Lion still does well on Clang. It's not perfect. Um, it's hard to find a tool that's perfect. Um, but this is a 1.0 of C-Lion and for being 1.0, for how well it works, it is working really well. Now, their extract method isn't doing that kind of, they call it a structural compare, right? When you extract a sequence of tokens and then it recognizes that that sequence of tokens exists elsewhere. And it says, you've just extracted that into a function do you want me to replace the sequence of tokens that appears elsewhere with a call to that function that's a structural

compare right um and it is doing extract method and rename and a bunch of other refactorings. You can see the results on my GitHub repository. And it's doing very, very well. And the only tools that I know of that do better are the fully automated tools that are supplied with clang and that's a different philosophy of refactoring that's kind of more batch map reduced style refactoring because that's what google does google has millions of lines of c++ it doesn't load into any known IDE. It's too big for that. And so they treat refactoring on their C++

codebase as a map-reduce problem. They map the changes for refactoring to all their source files and then they collate that and reduce it and apply it in one big swallow. And it's staggering to think of operating on a code base

that huge and doing it that way but they have no other choice if they want to keep the technical debt out of their code that's the only solution that's available to them their code base otherwise would just rot you know from changes that need to be made but are so burdensome to do manually that nobody's going to do them manually. because it's using the real compiler to analyze the source language and turn it into a data structure, that the transformation can be proved correct and therefore can be trusted. So the single biggest problem with a refactoring tool is how much can you trust it? It does me no good to automate my code but corrupt its meaning along the way. Automate transformations to my code is, you know, if you change the meaning, then I lose trust in that tool and I don't use it,

and then I'm just stuck with rot or doing it all by hand. And neither of those is a pleasant alternative. So let's talk a little bit more about automated refactoring. In that C++ Now talk, you actually created your own kind of simple automated refactoring tool using Clang technology. Yes. So the main difference between Clang and GCC, they're both open source C++ compilers with high levels of conformance of the current standard. Millions of lines of code get compiled through them every single day. They fix their bugs.

They have active communities. But the main difference between those two as a piece of software infrastructure is that Clang is designed to create libraries that provide the functionality of the compiler to allow other people to use those libraries to build tools on top of the parser and lexer and preprocessor that are part of Clang.

And that means that if I want to write a refactoring tool, and I'll get to why you might want to write your own refactoring tool and you might i'll get to why you might want to write your own refactoring tool right now it sounds like an esoteric thing that only like super refactoring freaks like me are interested in but there's a reason why every person who has a large code base would be interested in it and we'll get to that in a moment but clang acting as a library gives you access to the quality lexer that takes characters and turns them into tokens, the quality preprocessor that understands this sequence of tokens came from a macro expansion, and that macro was defined over in this file at this line, at this column, and its macro was instantiated in this other source file at this line.

And when working with C++, understanding where a token came from, did it originate as raw text in the source file that I'm processing? Did it originate as a result of macro expansion? And if so, did that macro come from a user-defined file or did that macro come from a system header? That's all very important in understanding what is a legal transformation you can apply to the code. And Clang has libraries that chew on your source text and give you all that information with enough richness and detail that you can answer all those questions. And therefore, you can make a tool that is guaranteed to do the right thing. Now, that's not to say we always write perfect software, but as far as we are able to imagine the kinds of things that our tools should do and write tests for them, we can verify that it does those correctly. Now, that all sounds well and good, but like,

why would I care about any of this stuff as just an ordinary application developer? I'm working on that big code base. Really good example is you're doing things like, say you're using the Qt library and you notice that Qt has a, so for those of you who don't know Qt, it has a string type called Qstring

and constructing a Qstring gives you an empty Qstring. gives you an empty QString. But if you say QString S equals quote-quote semicolon, now I've wasted time by taking this raw char star string literal and assigning it to a QString create an empty q string but just constructing an empty q just constructing a q string gives you an empty q string right okay so this is just wasted work right and it and as more people come into your code base and they see this all over the

place they look at that and they're like i am not that familiar with qt this looks like the way you're supposed to create empty strings right so they compound the mistake by writing more instances of this sure so you might say uh you know i'm the senior guy here on this team and i'm tired of seeing these junior guys make this mistake and we're gonna have a a big meeting about it. We're going to make everybody feel really guilty about doing this. Okay, but that's not a psychologically viable approach for most humans. It's not really that effective. What's better is if we could create a tool that'd just be like every night, just ran over your whole code base, looked for these and just said we just replaced q string s equals quote quote with q string s just just simplify it and we can trust that

transformation so we just run it every single night runs as part of the nightly build it fixes all those it commits them and they just permanently disappear from your code base and it doesn't matter how many junior engineers add that in the next day it will be gone the following night and eventually they'll just they'll just yeah they'll just be looking at the code and they'll be like huh looks like you know how come that thing took my s equals quote quote away and you go ask bob about it and bob says oh well it's redundant you don't need to do that and don't worry about it because we have a thing that just cleans that. It just mops the floor every night. It just takes care of that stuff for you.

But eventually they'll just be like, huh? Every time I type it, they're just going to take it out. So why should I bother typing it anymore? And then they just stop doing it. Now, so you would actually advocate having your automated build environment do

check-ins then? If you can trust it. Now, for that transformation that we just discussed, I mean, that's kind of simple and straightforward enough that everybody would be like, yeah, I could trust that, right? Right. That you could do with a RegEx almost. Yeah, but I bet I could find tricky ways that your RegEx wouldn't match.

Oh, I'm sure. Because all we got to do is start throwing the preprocessor in there and then suddenly it starts getting crazier. Right. But yes, if the transformation is 100% trustable, then, yeah, I would have no problem with having an automated janitor that would just come and mop that up every night. Now, there are some tools that ship with Clang that have certain kinds of transformations

that are trustable enough that I would be perfectly fine with running them every night. An example is there's a tool called Clang Modernize. That one I've used, used yeah and it has a very reliable transformation that replaces uses of zero or the null macro with the null putter keyword yeah we actually had that break code and one test in one case in the large code base I'm working on. Now, when it broke your code, was

the error in the transformation that was applied, or was the error that the code was wrong, and the error in the code was exposed by running the transformation? It misinterpreted something as a pointer that should have actually

been zero, but I wasn't the one who was deploying the transformation i was just looking over the shoulder of the person working on i cannot remember the details so we're we're fundamentally coming down to the issue of trust right how much trust can i put in one of these tools um and to establish trust my recommendation is run the tools manually and look at what the output is right examine enough cases to convince you that the tool is trustworthy then you can decide am i going to do this as a one-time deal or do i have new programmers hiring on into my team who keep reintroducing the very things that I got rid of? And then we have to decide if we want to run it.

You know, we'd run it once a quarter. And we kind of slog through the diffs that it generates. And, you know, or we just say, you know, that's too burdensome. I'm just tired of reviewing all these changes. They're all the same. And so far, I've never seen the tool get it wrong. So I'm just going to blanket give it permission to commit. Right.

Right. So you have to establish with any of these refactoring tools, whether they're invoked by the IDE through a keystroke or whether you invoke them kind of batch oriented. You have to decide based on actual usage on your code, how trustworthy they can be. And that's why I created that test suite and tried to publish results. If I feel gung-ho enough to run all those tests manually against some tool on a new IDE, it's because I was using a particular product.

The product is discontinued, so there's no point in mentioning its name. But it was horrible. It introduced so many errors into my code that I had to manually fix up every automatic transformation is what it seemed like. So that's why I created the test suite. So we could use that to say, like, well, looks like it's doing pretty good for these cases. And, oh, it's pretty good at renaming classes and variables with local scope, but it has a hard time renaming namespaces. We just can kind of look at the results for those test cases and say, okay, so it looks like it's high quality as long as we're just renaming local variables, renaming instance variables.

It's good at that. Renaming classes sometimes gets a little bit confused. That'll help you judge where and how much trust you should place in a tool. You've mentioned your test suite several times, by the way, but I don't think you've actually given the name of it, if we could have that. So my GitHub account is legalizedadulthood slash and the repository is slash refactor-test-suite.

Okay. Yeah, Richard, I feel like we could pick your brain for a lot longer about refactoring, but I feel like we might be running out of time. Sure. Do you want to put in a plug for your C++ user group? Sure. I had a lot of things I thought were cool were happening in C++ land,

but it wasn't appropriate to talk about it at the.NET user group or the Java user group. And we have CodeC camp here in utah but the event has become so large that it um is difficult to to organize more frequently than once a year and although i'd been presenting c++ topics at code camp over the past couple of years i wanted a more regular outlet so i started a utah c++ users group, and we've had mostly talks by me. We had one talk on 0MQ that was given by one of our members, which 0MQ seems like a really interesting library for distributed systems, by the way. That's a message queuing system, is that right?

It is. And it's kind of adopting the c++ philosophy of don't pay for what you don't use interesting so it's got uh you know higher level layers that you you pay for more stuff but they have more policy but it also has lower layers that you know you pay for less but you have to pick decide the policy yourself. And so there were a lot of topics that I thought were really cool and exciting. I'm a fan of, I mean, if I'm into compilers, I must be into parsing.

I think Boost Spirit was a really fun library to play with. And that was actually one of our first presentations. I want to see more people doing test-driven development in C++, so I talk about that. And I keep a list of possible future talk topics. And it seems every day I come across a new thing. I mean, there's just that Facebook Futures Library. That looks pretty cool to play with. And your next talk is actually about jason's

library is that right it is and that came directly from this podcast i was listening to the podcast i'm like hey that seems really cool um where i work at dad 3d we use qt for our primary application and qt has scripting support through uh embedding the javascript engine that's used in chrome yes and our application is completely exposed to script it's probably one of the largest scriptable qt applications on the planet it is very scriptable and very customizable and um that is that ability to take that application and extend it into areas that it wasn't originally intended for simply because it wasn't a feature that we thought about or a use case

that we had considered but it was important to somebody they were able to take our application and push it in that direction because of the scripting. We have a plug-in SDK is available as well where you could write C++ code as we know, scripting is a lot more approachable than C++ for most people. So we, you know, it's our applications highly extensible, but it's a lot easier. We have a little scripting IDE in the application. Actually, you can, you know, you can debug your JavaScript and set breakpoints and set breakpoints and execute it with keystroke and all that kind of stuff and see the application response. So I know that scripting, particularly our application is a content, what they call a content creation application.

So people primarily bring in 3D models and orient them and compose a scene and then render it in our flagship application called DAS Studio. But we also have other applications that are content creation applications. We have Bryce, which you might be familiar with. It's a kind of longtime contender in the area of landscape generation and generating environments. We have Carrera, which is kind of a kitchen sink, you know, modeling and animation and rendering package all in one. And then we have a pure modeler called Hexagon.

And it's a subdivision surface modeler for creating characters or creating props and things like that and uh there are for a lot of these applications they can be driven much further into the application domain with scripting so when i saw about kai script i was like wow this looks really cool and it's an easier way to show people how to add scripting to their application than saying oh oh, well, you just take the whole Node.js thing and compile it and link it in. I mean, it's like the idea of like, no, here's just like a header-only library

and you just include it and use C++, modern C++11 techniques, and it is designed to attach to your code in a natural and straightforward way. I think that's, not that there's anything wrong with the ECMAScript engine in Qt. I mean, we use it to a great extent. But for somebody to approach the idea of adding scripting to their application, that's a lot more intimidating than Kiscript.

So I think that's going to be a great topic to present on. Well, I'm glad I gave you something to talk about anyhow. Very cool. So where can people find you online if they want to find more of your work richard so sure i'm on twitter but i couldn't get all the letters in my name so it's legalize adult hd and no that's not high definition pornography that's just because i couldn't get enough letters i had to drop a couple and people are welcome to email me my email address

legalize at xmission.com I have a blog it's legalize adulthood dot wordpress dot com I have a blog that's mostly like interesting C++ news that I stumble across

that is for the user group and that's utahcpp.wordpress.com. And I'm Legalized Adulthood on GitHub as well. Very cool. Thank you so much for your time. You're welcome. It was a lot of fun. Thanks. Thank you. Thanks so much for listening as we chat about C++. I'd love to hear what you think of the podcast. Please let me know if we're discussing the stuff you're interested in,

or if you have a suggestion for a topic, I'd love to hear that also. You can email all your thoughts to feedback at cppcast.com. I'd also appreciate if you can follow CppCast on Twitter and like cppcast on facebook and of course you can find all that