CppCast - Testdriven C++ using Catch

Episode Date: May 27, 2015

Rob and Jason are joined by Phil Nash to talk about C++ Unit Testing with Catch. Phil is a semi-independent software developer, coach and consultant - working in as diverse fields as finance, agile coaching and iOS development. A long time C++ developer he also has his feet in C#, F#, Objective-C and Swift - as well as dabbling in other languages. He is the author of several open source projects - most notably Catch: a C++-native test framework. News CppCon 2015 Registration Open Cling Aims to Provide a High Performance C++ REPL C++Now 2015 Program is online C++Now 2015 Presentations C++ Now Youtube Channel New C++ experimental feature: The tadpole operators Poll: What C++ Testing Framework do you use? Phil Nash @phil_nash Level of Indirection Extra Level of Indirection Links Catch MeetingCpp talk on Catch ISO C++ Standard Discusson on Names C++ Extension Methods Sponsors

This episode of CppCast is sponsored by JetBrains, maker of excellent C++ developer tools. Listen in for a special discount code later this episode. And by CppCon, the annual week-long face-to-face gathering for the entire C++ community. Get your ticket now during early bird registration until June 10th. Episode 13 of CppCast with guest Phil Nash recorded May 27th, 2015. In this episode, we discuss the C++ REPL and the Tadpole operators. Then we'll interview Phil Nash about unit testing in C++. Phil will tell us about some of the features of his library, K only podcast for C++ developers by C++ developers.

I'm your host, Rob Irving, joined by my co-host, Jason Turner. Jason, how are you doing today? Doing great, Rob. How are you doing? Doing pretty good. We both submitted for CppCon this past week, right? Do you want to talk about that for a minute? That is correct. I submitted three talks myself, I believe. We'll see what they accept. How about you? I did submit one talk on cross-platform mobile. This would be my first big conference,

so I'm excited to see if they accept my talk or not. But yeah, I'm looking forward to it either way. Did you submit the same talks you did for C++ Now? I submitted an updated version of one of the talks that I did for C++ Now, the thinking portable talk, and two other smaller half-session talks just to see what the possibilities were. Okay, sounds good. At the start of every episode, I like to read a piece of feedback when we have it and this week we have a piece of feedback from Richard he writes in

your podcast is great I've listened to the first couple of episodes so far but then he asks why does your background image prominently show this not C++ code at the top and he goes on to say it looks like C++ CLI or C++ CX

code and points out that he sees this one piece of code that says throw ref new exception. Ref is obviously not a standard C++ keyword. It is C++ CX. When I first put together the background image, I just grabbed some random

sample that I had on my machine from MSDN and it happened to feature some C++ CX because it was a Windows sample. So after having Kenny Kerr on last week, I decided to go ahead and make a new background image after talking to Kenny and then getting this feedback from Richard. So if you look at cppcast.com now, you'll see that all the code in that header and footer image are standard C++, no CLI or C++ CX. And it's actually, I think, coming from the Cinder library, which looks like a pretty good source code base, and it's a project that I actually have been meaning to try out myself.

But I really appreciate the feedback, Richard, and if you would like to write into the show, you can email us at feedback at cppcast.com. So moving on, I want to introduce Phil. Phil Nash is a semi-independent software developer, coach, and consultant, working in as diverse fields as finance, agile coaching, and iOS development. A longtime C++ developer, he also has his feet in C Sharp, F Sharp, Objective-C, and Swift, as well as dabbling in other languages. Phil, welcome to the show. Hi, Rob. Hi, Jason. How are you doing today?

Yeah, I'm pretty good, thanks. Great. So before we get on to the interview, I just wanted to talk about a few pieces of news. The first one is, as Jason and I just talked about, CPP Con. Registration is now open. The call for speakers closed this past Friday, and they're now selling tickets. So I think there is an early bird discount if you want to get on there now and uh you know sign up for the show and uh do you have any plans on going to cvp con

this year phil well i did submit a talk proposal so we'll see if i get in on that probably we can't afford to go otherwise but uh for me it's an international travel right right yes yeah and they're um i'm trying to find the price do you guys see what the price is of the early bird discount i'm trying to see where that is i know last year that the early bird registration was a significant discount but i had not looked at it this year because also i'm hoping that my talk will be accepted right okay i found it so early bird registration is from now until july 10th and that's for 845 dollars and i don't see what the price is going to be once early bird expires but i would assume it you know going to go up a couple hundred dollars probably. So definitely sign up now if you have any interest in going to see

BBCon. It should be a great show. And it looks like Bjarn Strusup is going to be doing one of the keynotes. So that's great. Next piece of news is how Kling aims to provide a high performance C++ REPL.

Did you guys have a chance to look at this one? I had a look. I haven't had a chance to download and play with it yet, but I'm particularly interested in this, and it does actually tie in with what we're going to be talking about next, the testing, particularly TDD. The thing about test-driven development, when we talk about TDD,

we often say that it's not so much about the testing. That's almost a side benefit. It's more about the way it drives and shapes the design of your code, particularly the immediate feedback you get, not just in the results of particular inputs, but also how it feels to actually use the code that you've written, something that we don't often give too much thought to when we're writing it. Right.

And the interesting thing about a REPL is it gives you all of these same benefits. What it doesn't give you is the suite of tests at the end of it, so there's still a reason to do testing, still a reason to do TDD, but we have to sort of re-examine our assumptions about why we're doing things like TDD if we can get many of the same benefits. And I often use languages such as F sharp, which have good REPLs already. And one thing I've found is I very rarely do TDD in those languages. I still do testing, but I don't test everything.

Just some of the things I need to peg out against regressions. And it's going to be interesting to see with something like Kling whether I find the same thing with C++ now interesting i had not considered that i do some ruby developments i don't really do strong test different development but i do do a fair bit of unit testing and never really considered that the repl for ruby gives you a feel for the library and maybe you would mitigate the need to do some of the tests like you're suggesting yeah it's difficult to know how much of it's down to the repl and how much of it's down to

the language i mentioned f sharp with with functional programming of a different way of thinking about the code to start with so that may also be more of a factor in my case right yeah i've also used ruby and that's probably my first introduction to using a REPL. I find it kind of hard to wrap my head around what a REPL would really look and feel like in C++, because it's just so much more of a complicated language compared to something like Ruby. But it's definitely something I want to try out and see how well the REPL works with C++. So, Phil, did you get the chance to actually try Clean yet? No well the REPL works with C++. So Phil, did you get the chance to actually try Cling yet?

No, I haven't had a chance yet. Okay. I have some reservations about how it might work, but definitely keen to play with it. I've been aware of it for a while, but I've also never have tried it to use that. Okay. Well, it's definitely something to try out

for all of us then, and if you're listening and you try out Cling, the REPL, then let us know what you think. The next piece of news is about the C++ Now 2015 conference, which Jason attended a couple weeks ago. And it's not news, but they announced the program, so if you want to see who is speaking at that conference, you can do that. And the other interesting thing is they are announcing or presenting YouTube videos of some of the speakers. I think the only one they have up so far is Eric Kneebler's keynote on ranges, but I'm sure many more videos will follow. And you can also take a look at the slides used during some of the presentations at C++ Now. So that's definitely worth looking at. And this next article, Jason, do you want to talk about this one, the new C++ experimental feature? Yeah, if anyone is familiar with the old new thing uh blog it has

it's a an msdn blog that's been around for quite a while and um the the author here is mentioning a new experimental feature that visual studio is supporting called the tadpole operator which saves us the typing of instead of of doing y plus one, for instance, we might do minus tilde y and use the tadpole operator, or instead of y minus one, we can do tilde minus y. And it's an interesting article. If you haven't read it, I suggest you look at it and check out the comments also yeah i was going to point out the comments um it seems like a lot of people reading this article were very annoyed that microsoft was spending time working on this supposed new feature when they you know should

have been doing things like finishing up c++ 14 support um but the truth is, it's not a new feature. It's just this weird language quirk that exists, and he just kind of made up this story about it being a new feature that is going to help people. But it is something you can do with any C++ compiler if you feel so inclined. It looks like C Sharp and Java also works fine and pretty much anything that supports

binary math. Right, right. Do you have any thoughts on this one, Phil? Yeah, there's a couple of examples of things like this, such as the goes to operator that was going around a couple of years ago. It's like a long arrow. And you use it in a while loop. Actually, it's just the minus minus operator and then a less than or a greater than. Not new to operator overload abuse myself. I did a lightning talk a couple of years ago where I introduced the left arrow operator,

which actually allows you to do extension methods in C++, which is a great new feature. Unfortunately, actually, it involves overloading the less than operator on one object and the unary minus operator on another object. So it's a little bit awkward in practice. I wouldn't really recommend it, but it was fun to give the talk.

Right. You are why some coding standards documents say operator overloading is not allowed huh yep very good um so the next thing i want to talk about is i did a poll um since we are going to talk about testing frameworks today. I decided to put out this poll saying, what C++ testing framework do you use? And I put down a couple options, CPP unit, Google test, boost.test, catch, Aaron, cute, fructose. I'm doing a customer homegrown unit test method,

or I'm not doing any unit testing. And Phil, I wanted to know what you thought might be the most popular option i'm going to go with uh with google test possibly followed up by no unit testing uh close actually very close uh so google test was the number one option but you would be happy to hear that Catch was the second most voted for unit testing option, but it was tied with, I'm not doing

any unit testing. There's some room for improvement then. Let's do the poll again next week and see if that changes. That's a great idea. So yeah, there were about 34% for Google Test, and like I said 17% for Catch, and there were a couple votes for boost test and CPP unit.

Aaron Cute and Fructose did not get any votes, and a couple other people were doing homegrown testing or not doing anything at all. I have to admit, I had never heard of Aaron Cute and Fructose until I saw your poll. So I actually pulled those from one of Phil's slides on your Catch website, I think. That was one of the testing frameworks you pointed out, Phil? Yeah. I think some of those frameworks are not actively maintained anymore. Okay. It definitely seems like the Google and Boost test are some of the more popular options.

So let's start talking about that, though. So as it's implied by what we were just talking about, you have been working on a C++ test framework called Catch, and there are several other C++ test frameworks out there already. Why did you feel the need to make a new one? That is a very good question. There is a bit of backstory there.

Okay. And it starts some years ago. I was actually working in Objective-C, funnily enough. And at least back at the time, there was only really one Objective-C test framework in town, and that was something called OCUnit. And again, at the time, this has all changed since, it wasn't particularly well integrated with with xcode the the apple ide and it's a bit of a pain to use to be honest i didn't really give you a lot um for that pain and i just seem to remember c++ frameworks seem to be a lot better than this and one of the

interesting things about objective c is you can interoperate with c++ using a language called objective c++ sort of spiritually similar to c++ cli on the dotnet framework it's um allows you to freely mix objective c and c++ in the same source code so you can actually use a c++ test framework compile it as objective c++ and test your objective c code so i tried a few of them out some of the ones that i'd used before and they actually seemed to be worse than i remembered possibly because i was just primed for looking for the faults in them at this point maybe because i've been doing some c sharp for a few years but the one thing that really struck me was that they all seem to be at least spiritually ports of j unit into c++

so they sort of play to the strengths of java and you know to some extent c sharp but didn't struck me was that they all seem to be at least spiritually ports of JUnit into C++. So they sort of play to the strengths of Java and, you know, to some extent C Sharp, but didn't really play to the strengths of C++. They just sort of lost things. So as any typical programmer would do, I thought, I'm sure I could knock up something better than this in a weekend. So, well, I actually fought that urge for a while, but sort of an experiment on the side, I started playing with a couple of ideas I had. so well i actually thought that urge for a while but sort of an experiment on the side i started playing with a couple of ideas i had um one of the early things i wanted to do was to try and

capture the the left hand side and right hand side of a binary expression get to get the values of them without having to rearrange your your logic so most traditional frameworks you'll have to write something like assert equals a comma b when actually what you want to write is a is equal to b and assert on that and i sort of got something going but it sort of used two macros and it was a bit ugly um i wasn't too happy with it and then went to a talk given by kevlin henney now if you ever get a chance to see a talk by kevlin henney you should take it you ever get a chance to see a talk by Kevlin Henney, you should take it. You'll always learn something and be entertained at the same time. This was no exception.

This talk was about C++ unit testing, and he actually presented a technique that he had come up with using expression templates to do exactly what I was after. You should decompose an expression and get the left-hand side, right-hand side values while maintaining a natural syntax. So, of course, I immediately stole this with his blessing and put it into my own framework. In fact, there was another feature from Kevin's talk that I used as well, something that I now call sections, which is a way of effectively embedding test cases within test cases so you can have nested sections within a test case and so each leaf section is executed sort of in isolation it runs a whole test case from the start again and the the consequence of this is you don't really

need test fixture classes anymore another one of the sort of the J unit ways of ways of looking at things much more natural way in C++ is to set things up on the stack and then share that between multiple sections so again you know this is a much more natural C++ way of of testing that I haven't really seen in many other frameworks so that's um that's really what got me started at that point i i realized i had something of value to share put it out onto onto github and uh that was about five years ago and i've been actively developing it since oh wow since you started with the the the problem you were trying to solve was xcode integration i'm curious did you solve that problem? Yeah.

So I did use, well, I still use Catch for testing Objective-C, even to this day. I'm not actually doing quite so much iOS development now. What I am doing, I'm doing mostly in Swift. But in the intervening years, first of all, OC unit's got much better integration into Xcode. Now Apple fully support it. But also a couple of new frameworks have come along that do most of what Catch was trying to do for Objective-C, but again in a more natural Objective-C way. So I've sort of concentrated now more on the C++ side, which is where it seems to be a better fit.

Okay. So what's the process like to first get started with Catch? How long does it take to start writing new tests using it? Yeah, this is one of the pain points that I found when I was looking at other C++ frameworks, particularly Google Test. That was one that I really wanted to get running because it seemed to be very popular, but it took me about an hour to to actually get it up

and running in xcode originally and that just seemed too long and i think a lot of people get put off by just how long it takes to get into one of these frameworks and they end up not doing any testing so i wanted to make that initial barrier as low as possible so i'm really focused on trying to get that as simple as possible uh one of the talks I do on Catch, the very first demo I do, I'll go to GitHub, download Catch, which is just a single header file now, include that in a brand new source file,

write a test, run it, debug it, and the whole thing is a couple of minutes. And that blows a lot of people away that you can actually get up and running so quickly with it yeah that's really impressive um does catch rely on any modern c++ 11 or 14 features yeah that's another really good question uh i wrote catch originally say about five or six years ago now right before we had c++ 11 at least it wasn't called c++ 11 then so i didn't rely on any c++ 11 features at all at the time um and to this day i don't rely on c++ 11 but i've had to support a number of features um for example a null pointer i've needed

explicit support for a few other things internally like no accept and deleted methods just to to avoid warnings more than anything else but i've avoided actually using any c++ 11 features so far because i don't want to alienate people that still use c++ 03 compilers but one area that i'm considering using it, conditionally at least, is the threading primitives. One of the number one requests that I get about Catch is, does it support multi-threading? Well, it's really difficult to do that in a cross-platform way with no dependencies. But C++11 now has threading primitives that I could use to offer that. So that's one area I might go.

Since you brought up cross-platform, what compilers do you support? What compilers do you test against? Yeah, that's two different questions. Sorry. No, no, that's good. I primarily develop Catch on my Mac. So I'm using Xcode, which is Apple's own version of Clang.

Right. I do sometimes test myself against GCC, but mostly I just pick up reports from other people. And then in my day job, I'm using the Microsoft compilers. At the moment, that's Visual Studio 2010. We're also using 2013. So they're my primary platforms. but by trying to target standard c++ 03 as much as possible that theoretically you know shouldn't be any limit to what

compilers are supported i don't use any platform features at least only conditionally okay very good um what types of output can you get from tests in Catch? Is it only going to be console output or there's support for other types? No, the output is another area of quite intense focus in Catch and I really believe that the

richness of the output you get is going to impact how useful the framework is going to be to you. So I try to make it as simple as possible to get the richest possible output. So by default, everything goes to the console. So I make that a first class reporter environment. So you get your file and line number, the failing test, the expression that failed. So you see the expression as it was written.

And then you also see the expansion because i captured the left hand side of right hand side and because it's showing you that you don't actually have to worry about what was the actual and the expected value it's one of the things i always get confused when i'm using other other frameworks but as well as the console reporter there's a modular reporting system so So there's an XML reporter, a JUnit reporter, a TeamCity reporter, but you can write your own reporters and plug them in.

And a number of people have done that. So it's quite flexible in that regard. Wow. I didn't realize it had that many different possibilities for output. I'm curious about the console output. How does it try to mimic the msvc format or the gcc format or whatever so that the ide that you're using maybe you would be able to parse it and you couldn't click on it and jump to the code that

failed or does it do anything like that yeah so when it reports the file and line number it does that in the msvc format when you're using that and uh for gcc and clang it does it in in their native formats so you'll be able to just double click and go straight to the file and line number very good um it also when it writes the word failed it puts it on the end of the line which looks a bit. The reason I do that is because in GCC and Clang, that will actually show up in certainly in Xcode as a test

failure. It's got built-in support for that. You'll see a little red dot next to that line of source code. So that's quite a nice little extra feature there. So if you don't mind if I maybe dig in just a little bit here, have you had any problems with, um, trying to guess which compiler you're running on and make

sure you do an output that's appropriate, maybe regressions, like you make the output better on GCC, but that somehow broke something on MSVC. Just curious about the development process that you've had to go through. Yeah. I haven't had many problems in terms of the word to the output. Once I got those two things down, it pretty much worked. I also do colorization, which obviously works different ways on Windows and POSIX environments. That's been working pretty well. But there are certainly other challenges when you're going cross-platform,

detecting certain compiler features and platform features, and particularly bugs and different ways of doing warnings. And then I'm really reliant on the community to feed me back when they hit problems because I just don't have the facility to test on so many platforms. So it seems like a constant battle to particularly fix warnings that show up in different platforms. Yeah, I've actually used Catch a little bit, and I have noticed a couple of warnings that I can imagine it's difficult to keep track of them all. Yeah, well, let me know. Okay, I'll do that.

Yeah, I do find that it is quite annoying. Sometimes a warning on one platform, you fix that, that causes a warning on another platform. They're sort of mutually exclusive. Yeah, sometimes, yeah. You have to do a local disable of the warning if it's something that's just not fixable. Yeah, I try to keep those to a minimum,

but they're definitely building up. Yeah. I want to interrupt this discussion for just a minute to talk about this special offer that JetBrains has made for CppCast listeners. JetBrains makes some awesome tools for C++ developers in any environment. There is the ReSharper C++ plugin for Visual Studio developers, AppCode if you're working on iOS or OSX apps, or their new cross-platform C++ IDE, C-Line, which runs on Linux, Windows, and OSX. JetBrains is offering a coupon code which can be used to get a personal license to any of those tools for 25% off. The code is cppcast jetbrains cpptool. All one word, just enter that code during checkout where it prompts you for the discount code.

Again, that's cppcast, JetBrains, cpptool, which will get you 25% off any of the JetBrains C++ tools, C-Line, AppCode, or ReSharper C++. JetBrains has extended this offer to CppCast listeners until June 30th, so head to jetbrains.com cpp and take advantage of this great offer uh so what's the future of the catch library look like are there more features you're planning on yeah as i say it's inactive development um and that's not just bug fixing but i'm adding sort of small and large features as we go um there's a few larger features that i do have planned and i don't like talking too much about where it's going because some of these things i've been talking about for years i still not got there yet but um i already mentioned the threading support that just

seems to be too important to ignore so i will be looking at that at some point another one which has been in the works for a few years and i've got a little sort of beta implementation but haven't progressed much beyond that yet, something I'm calling generators, which is a way of doing property-based testing or generating lots of test values and potentially expected results. So you can write one test body that will then test against hundreds or thousands, maybe even millions of values at a time.

You can also even do the cross-product of multiple values. So that's something that I have a version of at the moment, but it's not fully featured. I wouldn't advise actually using what's there at the moment, but I do want to get that right. And another one that's had a stop implementation in there for a while is matches. You may have come across matches before, sometimes called a ham crest on some platforms. And this is just a way of writing assertions.

It can be much richer than just comparisons. You can say, for example, that a string contains a certain value or begins with a certain value, or a container has certain elements in it. And this then gives you very rich output, makes it nice and easy to test in quite complex conditions, and is extendable by the user, usually. So I'm still undecided at this point whether I need to build a fully-fledged Matcher framework into Catch, which would be quite a big job, or just make it easier to interoperate with an existing Matchers library.

So that's definitely another area of active interest. Sounds like some of the things you're talking about could be facilitated with standard algorithms, but not all of them. So I don't know. Yeah. For the matchers. It's a similar idea, but matches need to be written in such a way that they can interoperate with the way I report on test failures. Right. report on test failures right that's quite an important part of what they do actually reporting

when when something is going wrong exactly what has gone wrong okay that's um they both sound like very useful features to me so i'm sorry go ahead i was going to say they're very useful but but quite big jobs as well. Sure. Yes, sure. You've spoken about Catch multiple times at conferences. Is that correct? I have, yes. In fact, the last such occasion was at a meeting CPP just last December in Berlin.

And that one's on YouTube. I think I sent a link to you that you can put in your, your show notes. So you can see me talk about it there. I'm also going to be talking about it again. It's a NDC Oslo. It's just next month. And I did submit to CPP con as well.

So I haven't heard back yet whether that's been accepted, but it may end up seeing you guys there. Yeah, actually, I think we get told in like two or three weeks. Yeah, that sounds about right. Proposals are getting accepted. Yes. I saw your Berlin talk, meeting CPP talk, and that was a talk that actually led me to start using catch in my projects. And I have greatly appreciated how it works.

It is very natural to use. That's good to hear. Thanks. So I do recommend that video if anyone hasn't seen it yet. Check it out. Yeah, and I'll put a link in the show notes for the meeting C++ video. I watched that one as well. It was very good. I go into a bit of detail on that, on how to do the expression templates as well. So I know a lot of your listeners

will be quite interested in that. Yeah, definitely. So is there anything you would like to talk about? Kind of more general TDD practices for C++ developers? I see you've done some more general talks on TDD in the past.

I'm looking at your speaker history right now. Yeah, actually most of the talks I've done on TDD have been in an iOS context. Okay. Actually, iOS and C++, they're similar in that they seem to be underrepresented when it comes

to testing and TDD, which is why it's an area of interest for me uh you know i've been looking at um why that might be a lot of it's down to the community just haven't really picked up the ideas but also some you know technical limitations and technical obstacles which is again one one of the reasons i wrote catch was to try and overcome or minimize some of those uh So I do think making that as easy as possible is quite a big aspect to it. But I think particularly with C++, just things like build times and other issues with the whole sort of test and see the output cycle, It's much longer than it is in other languages. And that can really impact how much value you get out of it.

There are ways to mitigate that. Again, you know, using a framework that makes things easier is a big part of it, but making sure your build times are fast, or certainly incremental builds is also very important or um making sure that you're not trying to test too much in one go you're going to try to test a big sort of end-to-end piece of your system rather than just an isolated unit that you pull out and that's much easier when you start off with the test when you do tdd rather than trying to add tests later.

But it is hard. It is an area that a lot of people struggle with. So it's something I'm still looking at. I also saw in your meeting C++ talk, you went into the concept of behavior-driven development, which is related to TDD, I believe. Could you go into that a little bit? Yeah. So BDD, behavior-driven development. So originally, this was just TDD, but done well. That's how it was described. Added a little bit of format and rigor around it. In particular, one of the things that came out of that was this idea of

writing your tasks in a given when then structure. So you say, given, you know, these set of this state, when I do this thing, then, and then you write your expected result. But by framing it that way, you actually write down the sort of natural language description of what you want to see. And I've incorporated this into Catch, sort of overlaying it over the top of the sections macro. So you can say given, and then a section name, write your code, when a section name, write your code, then, and then your assertions. And the nice thing about that is when you see the report on it, you'll actually see it laid out nicely in a way that you can just sort of take away, give to someone non-technical, and they will see what that test is about

and maybe even make sense of what the output's telling them. But BDD itself has sort of moved away from that purely technical side, and it's become more about having the non you know having the non-technical conversations making sure that you get the requirements right to begin with and that's obviously really important no matter how much testing you do no matter how good your code is if you're writing the wrong thing then it's it's all for nothing you're just doing the wrong thing faster so it's a bdd is about having the conversations with the non-technical people who can tell you when you're trying to write the wrong code.

Great. That makes sense to me. One last thing I want to talk about with Catch was another thing I saw in your meeting C++ talk is it was really easy to tag different tests so that you could run different sets of tests under different conditions. Could you go over that a little bit? Yeah, so a lot of test frameworks have a way of putting together test suites where you can put a set of test cases together, say this is a test suite for testing a certain aspect of the system, say. And you might classify them according to a part of the system they're testing

or also characteristics such as whether they're long-running tests, whether they're more sort of end-to-end tests rather than purely unit tests. And Catch does this by letting you tag test cases, just a free-form string you put in square brackets. And then you can run tests with certain tags or certain combinations of tags, or even the absence of a tag. So one of the things I do is I will tend to tag my integration tests,

which are not pure unit tests. They will touch the file system. They may touch databases. They may test larger parts of the system and be longer running. But they still work well in a unit test framework. By default, I'll hide them. There's a special tag,

this dot, that means that when you run your test executable with no arguments, it won't run those hidden tests. And then by adding the integration tag as well, I can easily say, now run run all the integration tests so now with a single executable i've got unit tests and integration tests in there and i can easily select which ones i want to run so on a build server i'll always run all the unit tests then all the integration tests and also do things

like performance tests and maybe other types of tests that way um and in some cases there might be some you know much slower tests that i want to exclude even from that so it's quite flexible what you can do with it i can definitely see how i could use that you know at my day job i'm actually currently going into more of a test driven development uh methodology and right now we're using something homegrown but i'm definitely At my day job, I'm actually currently going into more of a test-driven development methodology. And right now we're using something homegrown, but I'm definitely going to look at implementing Catch because it looks like a really great library. And it's great work that you've been doing. Is there anything else we should go over before we let you go? No, I think we covered all the things that i had in mind jason did you have a

question uh no but if you don't mind for a moment i'd like to turn things around on my co-host rob here and bring up a blog post that he recently did called uh thoughts on dark matter developers and i just it seemed very appropriate for the for the podcast I don't know, Phil, if you looked at this, but if you had a second to talk about your thoughts here or not, that's up to you. Yeah, so if you're not familiar with the phrase or the term dark matter developers, it's a term that Scott Hanselman coined in a blog post he wrote

a couple years ago, I think. And it basically refers to any developer who is not going to conferences or user groups or engaging in the community, whether it be online, virtually, or going to user groups and things like that. And I don't know, it seems like C++ has a lot of Dark Matter developers. And I think that might be slowly starting to change, and I hope it continues to change. Part of the reason why I started this podcast was to maybe try to reach out to some of them. Yeah, that's about it.

I just hope to see more C++ developers join the community, because I'm enjoying the community, and I'm glad to be a part about it. I just hope to see more C++ developers join the community because I'm enjoying the community and I'm glad to be a part of it. Alright. Sounds good. Yeah, I just had something to say about that. Being sort of part of a number of different developer communities

not just C++, I've really noticed how much of a difference the community makes to the way you engage with the environment or the language, but also just how you feel about the work you're doing as well. And when you've got a good community there and you know who to turn to, it really makes a big difference. So really good work you're doing with the podcast.

I appreciate it. And I really appreciate the work that the Meeting C++ group has been doing in encouraging new C++ user groups and that John Call was doing with CppCon and bringing C++ talks to code camps. So I'm only doing a very small part, but I'm really glad to be a part of this community. And if you're listening, please share the podcast

and let your friends or C++ developers know about the podcast and also about user groups and everything else that's out there. So, Phil, I guess we already asked you if there's anything else you wanted to go over. I know you have a great blog. So where can people find you online if they want to read more about Catch or more about the other things you're doing? So I've got a forwarding URL for Catch. It's hosted on GitHub, but if you go to catch-lib.net, I think this will be in the show notes, that would take you to the GitHub site.

And from there, there are links to – there's a Google Groups forum. It will also link back to my blog. But my blog is levelofindirection.com. And if you find that hard to remember, I've also got extralevelofindirection.com, and that redirects it. Yeah, I love that URL. That's great.

On Twitter, I'm phil__nash. Don't try Phil Nash, because that's another developer that works in London that's often confused with me, and he doesn't like it. Okay. Well, thank you so much for your time, and thanks for talking to us about Catch. Thanks for inviting me on the show. Great. Thanks so much for listening as we chat about C++.

I'd love to hear what you think of the podcast. Please let me know if we're discussing the stuff you're interested in, or if you have a suggestion for a topic, I'd love to hear that also. You can email all your thoughts to feedback at cppcast.com. I'd also appreciate if you can follow CppCast on Twitter, and like

CppCast on Facebook. And of course, you can find all that info and the show notes on the podcast website at cppcast.com. Theme music for this episode is provided by podcastthemes.com.