CppCast - Trompeloeil Mocking Framework

Episode Date: February 22, 2017

Rob and Jason are joined by Björn Fahller to talk about the trompeloeil Mocking Framework for Modern C++ Unit Testing. Björn Fahller is a senior developer at Net Insight, and has been developing software for a living since 1994, mostly embedded programming for communications devices. Björn learned C++ from usenet and the ARM (Annotated Reference Manual) which was the standard before there was a standard. On a hobby basis, Björn likes to find silly solutions to non-problems and to explore effects of programming constructs.   Outside of programming, Björn is a member of a small group thet brews beer together, and is also a member of a volunteer organization of aviators who help with things like search and rescue operations, forest fire monitoring, and storm damage assessment. News Multithreading with C++17 and C++20 Distinguishing between maybe-null vs never-null is the important thing Going Native 56: Cmake in Visual Studio Björn Fahller @bjorn_fahller Playful Programming Links Trompeloeil Mocking Framework Björn Fahller - Mocking Modern C++ with Trompeloeil Sponsor Backtrace JetBrains

This episode of CppCast is sponsored by Backtrace, the turnkey debugging platform that helps you spend less time debugging and more time building. Get to the root cause quickly with detailed information at your fingertips. Start your free trial at backtrace.io.cppcast. And by JetBrains, maker of intelligent development tools to simplify your challenging tasks and automate the routine ones. JetBrains is offering a 25% discount for an individual license on the C++ tool of your choice, CLion, ReSharper, C++, or AppCode. Use the coupon code JetBrains for CppCast during checkout at JetBrains.com. Episode 90 of CppCast with guest Bjorn Fahler, recorded February 22, 2017.

In this episode, we discussed the future of multithreading in C++ 17 and 20. Then we talked to Bjorn Faller. Bjorn talks to us about the Trompe-l'Eye Mocking Framework. Welcome to episode 90 of CppCast, the only podcast for C++ developers by C++ developers. I'm your host, Rob Irving, joined by my co-host, Jason Turner. Jason, how are you doing today? Pretty good, Rob. How are you doing? I'm doing okay. No real news on my side to report you.

Nothing at the moment. Okay, let's just jump right into it then. At the top of our episode, I'd like to read a piece of feedback. This week, we got a lot of feedback from last episode. I was going to read this one from Victor Stepanov, who left a comment on our website, cpcast.com. And he actually left a comment with adding to our show notes, because I guess we mentioned a couple sites and websites and resources while we were talking that he felt should have been in the show notes that we forgot to add ourselves. So that was really helpful.

Thank you, Victor. I did go ahead and add all of those links to the official show notes. And one thing I wanted to mention, which I already mentioned to Victor in the comments, was if you ever want to modify the show notes, you can actually do that. There's a button at the bottom of every entry on our website that says you can go do a pull request on GitHub. And it's just a Markdown file. It's a pretty easy format, even if you've never used Markdown before, and you can just modify a link and put in a pull request, and I'll get a notification about it and approve it, and it will get added to the website.

It's pretty easy. That's pretty cool. I hope people take advantage of that, actually. Oh, yeah. It's a great feature. And Victor will be sending your info over to JetBrains for the free license raffle giveaway. And thanks a lot for the feedback. We'd love to hear your thoughts about the show as well.

You can always reach out to us on Facebook, Twitter, or email us at feedback at cpcast.com. And don't forget to leave us a review on iTunes. So joining us today is Bjorn Fahler. Bjorn is a senior developer at NetInsight and has been developing software for a living since 1994, mostly embedded programming for communication devices. Bjorn learned C++ from Usenet and the ARM, which was the standard before there was a standard.

On a hobby basis, Bjorn likes to find silly solutions to non-problems and to explore effects of programming constructs. Outside of programming, Bjorn is a member of a small group that brews beer together and is also a member of a volunteer organization of aviators who help with things like search and rescue operations, forest fire monitoring, and storm damage assessment. Bjorn, welcome to the show. Thank you. Yeah, that's pretty cool.

You know, Bjorn and I have chatted a lot on Twitter over the last year, but I did not know that you were involved in search and rescue operations. Yeah, well, involved in is an exaggeration, but I have training for it and I've been on a lot of exercises, but it's a volunteer effort. And I've been on calls several times. I was called out on an alarm once, ever, but it was canceled while I was in the car on my way out to the airport.

No go. Well, I guess it's better to be prepared and not have to do it than the other way around. Definitely. So you do have a flying license, though? Yeah, I got my private pilot license in September 2001, which was pretty interesting.

Rules suddenly changed. yeah yeah oh okay so it was a little bit unclear for a while what what happened but yeah i got my license also that's very interesting huh and what kind of beer do you brew um it's a mixture but it's mostly ale type of beers. I haven't tried lagers. They are difficult to make. But it's a lot. As you mentioned when you read the intro, we're a small group that brews together. So we discuss what we want to make and we make something that we come up with. So it can be a stout or an IPA or a Belgian blonde ale or whatever.

Very cool. I tried that once. I have a little beer brewing kit. Probably nothing as complex as what you and your group does. Did you have the Mr. Beer kit, Rob? Yes. Yes, I did. It's a small kit you can get at a lot of stores in america for pretty

cheap like 50 bucks or something something like that and you put everything together and you leave it out for a while and you actually finish the process by putting it in the fridge and then you can just pour it out from there it's pretty pretty neat i've heard about those i've never tried it never seen it actually yeah anyway uh we have a couple news articles to go through bjorn feel free to jump in and comment on any of these and then we'll start talking to you about your mocking library sound good good okay uh so this first one is another article from uh rainier grim we've talked about several of his on the show i think and this one he's been publishing a lot of articles lately he has been putting out a lot of articles.

That's true, because I think we talked about one of his last week, too. And this one is multithreading with C++17 and C++20. And he's basically focusing in on multithreading related features that are being introduced in 17.

And I guess he already knows the future, because he's already predicting all the multi-threading features that will be in c++ 20 i'm not quite sure how he has such confidence on that since it's you know hasn't really started yet i mean we haven't even finalized c++ 17 yet but he's outlining what he thinks will be in c++ 20 i I, well, I mean, if we want to talk about predicting the future, I agree almost 100% that we'll get atomic smart pointers and standard future extensions, because I know that there are implementations of those already in the wild. But the, and coroutines that we have and implementation of it, the transactional memory, latches and

barriers, task blocks, I don't know enough about them to comment on that. Yeah, I guess coroutines is kind of the biggest feature that everyone was hoping would make into 17 so it seems likely that it should make it into 20 i mean microsoft already has a working implementation like you said and clang actually now too oh yeah so hopefully this will all make it in uh bjorn what do you think or bjorn what do you think about about this article? It was interesting. I really don't know enough to speak of about... I don't have that much experience with multi-thread programming to begin with, and I haven't really followed that part of standardization,

so I don't know really. One thing I did want to say, though, is on coroutines on its own, a really cool thing that I think replace those threads with coroutines instead, and suddenly you have something that is much better performing if you have a single-core CPU, which you probably don't. But you also have a situation where if you have a bug and you can reproduce it there, then your troubleshooting will be a whole lot easier. Right.

Yeah, that's a good point. Kind of removing yourself from the actual issue of the threading and just more thinking about the asynchronous nature, I guess. Yeah, and if you can reproduce a bug in a coroutine setting, then you know that whatever your bug

is, it's not a data race. It's a logical problem. Right. Right. That's a good point. Okay, this next one is a blog post from herb sutter and it's distinguishing between maybe null verse never null and i guess this comes from a discussion on the core guidelines library and he was just uh defending why they're doing not null, right? Yes. Yeah.

Do you want to talk about this one, Jason? Well, I read the article. I'm not so familiar with the argument, but it seems that it must have been. It must be quite the argument because the person that he's quoting says that valid concerns are being dismissed, essentially. And Herb assures him, no, we went to great effort to consider how we wanted to address these issues. Right. And goes into his defense of using not null and why it makes sense with various pointer types, right?

Yeah, and the discussion was, should it be not null or should it be maybe null, basically? Like, which the discussion was, should it be not null or should it be maybe null, basically? Like, which way do we want to prove this? And Herb's saying more than 50% of uses, he thinks, of bare pointers are things that might be null. So that's the standard usage, so not null

is the exceptional case. That's why we should have a wrapper for it. Okay, anything else to add on this one, Bjorn? No, I think Jason summed it up That's why we should have a wrapper for it. Right. Okay, anything else to add on this one, Bjorn? No, I think Jason summed it up quite correctly, as I see it. Right. Okay, and this last one was actually a video from Microsoft's Going Native series,

and this one was about updates they're making to CMake in Visual Studio, which I think we've talked about a couple times before on the show. And they went through a demo of it in the video, and it's starting to look pretty good. Jason, did you have a chance to watch it, the CMake support? I got like two-thirds of the way through the video before I went out to breakfast this morning, actually.

Okay. But yeah, you're now able to load up Visual Studio without having a solution, and it'll just load up using your CMake file. So if you are using CMake on your Microsoft environment and you make a change in the CMake list file, you'll no longer have that prompt you to reload your entire solution. It's just a better behavior when CMake is changed. So it looks like something that's good for CMake users on Windows.

I think for users who are CMake users, I think some of the questions that I had that they answered in the video are, there's a configuration file that lets you set the defaults that you want for the various configurations that it's going to build. It defaults to four configurations, x86 and 64-bit debug and release each. And they look a little bit more like what you would expect from a CMake project and a little bit less like what you expect from a defaulted Visual Studio project. I feel like the namings and stuff of those are. And it seemed you did not have to explicitly say, hey, this is a CMake project, I want to configure it.

It detects that it's a CMake project and configures it for you. Right. Yeah. And I think we're going to have a guest from Microsoft to talk about Visual Studio 2017 next week, I believe. Right, Jason? Is that when that is, or is it the week after? Someone's coming on soon to talk about that. I'm not sure. Bjorn, do you have any thoughts on this? Are you a CMake user? No, I'm a novice CMake user, and I'm aspiring to one day become a novice Visual Studio

user. I'm not the right guy. Let's start talking about uh the mocking library though uh first what's the proper way to pronounce uh it's written out trompe l'oeil but i'm guessing i'm butchering that horribly yeah okay uh he's my punishment for trying to be clever isn't it i'm butchering the pronunciation myself i i say i say trompe l' I say trompe l'oeil trompe l'oeil it's a French term so it should definitely be said

with a more throaty R that I think I'm physically incapable of producing but it doesn't really matter trompe l'oeil it's an term, and those tend to get their own lives in different languages. So, for example, you talk about film noir, which I'm pretty sure is not the exact French pronunciation. A little bit of background on that. Trompe l'art is sort of a visual pun, I guess you can call it, where the artist paints something that you see

and then you notice something in the painting and it makes you go, ah, okay, and then you see the painting in a different way. And the literal translation is trick the eye. So if you look up the hashtag TrompeLay on Twitter or Instagram, you will find a lot of really good works. Oh, interesting. Okay, so do you want to talk a little bit about what the mocking library is and what it's used for?

How it's different from other mocking libraries? Sure thing. Well, mocking in general, to begin with, the idea is that when you're writing a unit test, you have your unit under test that you want to

exercise and ensure that it does what you want it to do. And to be able to test it in isolation without bringing your whole program in, if you do, it can be difficult to monitor fine details and it can be difficult to give it certain sequences of events. replace all its collaborators with mocks. That is, you paint a picture of the environment that you want your unit under test to see, hence the name. You paint a trompe l'oeil for your unit under test. So it perceives the picture, an illusion of a situation

that you as a test writer have set up and then you poke it in different ways and see that it behaves the way it should. So in classic object orientation you represent collaborators by interfaces of pure abstract classes and in the tests you replace the real implementations with mocks. Now in C++, we obviously sometimes have templated collaborators that don't naturally use interfaces. So in your tests, what you do then, you instantiate your object or a small cluster of connected objects that together mean something.

And you pass in your mocks instead of the real implementations of its surroundings. And then you place expectations on the mock objects to say, when do you want a certain member function of a collaborator to be called with which parameters and then you can decide what should happen when that is called, what should you return

should you maybe throw an exception or even call the unit under test back recursively that is nasty a difficulty when doing this, it's a temptation is to call the unit under test back recursively. That is nasty. A difficulty when doing this, it's a temptation is to be much, much too close to the implementation and accidentally over-constrain the expectations. So it's one of the difficulties, and this is obviously completely

unrelated to which mocking framework you use, is to know how much you should relax the expectations. Because obviously, if you relax them too much, the test is kind of meaningless. But if you restrain them too much, then you can never make any changes to your implementation without failing tests. So that is the background on mocking in general. Was it anything more you wanted about that before going on?

No, I think that's a good overview of mocking. Yeah. So does my attempt to pronounce this, trompe l'oeil, trompe l'oeil, yeah Trompe-l'oeil, yeah. Trompe-l'oeil. Does it work with other unit testing frameworks? It's something that's designed to work with another testing framework. Is that right?

It's not designed for any specific unit testing framework. To the best of my knowledge, it works with any of them. I might be proven wrong, but I think so. What you want to do when you use Tromplay with your favorite unit testing framework is you look up the documentation. The first item in the Tromplay cookbook is how to adapt Tromplay to your unit testing framework. And there are a number of unit testing frameworks already listed there. And if you use any of those, it's just a matter of copy-pasting a code snippet into your test program and you're done. If you don't have any of those, there's also documentation for how to write an adapter. It's not difficult.

And if you do, please contribute a pull request to update the cookbook. By default, Tramplay reports errors by throwing an exception. But this is, to be honest, this is not really desirable. For one, a mock function is by definition called by your unit under test.

And so if the mock throws an exception to say that something is wrong, it's really bad if your unit under test catches that exception. You don't want that. And also another thing is that there are situations when Trompey must report an error during stack rollback and you really don't want to throw an exception at that situation.

This, by the way, gives me a reason to shout out to makers of unit testing frameworks. Please, please make them so that you run each test case in its own process. This has several great advantages. The most obvious one is isolation from tests it becomes more or less impossible to have tests that accidentally

affect other tests it also makes it more relaxed for you when you write tests because you only have to write code up to the point where you want to prove something show something and after that you want to prove something, show something. And after that, you just let it go. The process dies.

You don't have to worry about how you clean up the state for the next test. And one of my favorites is that you can be really brutal with failures. I prefer to call a border terminate because to begin with it doesn't matter if the error is reported during stack rollback or whatever. You terminate, that's it. You print an error message and you're done. If the error situation is not obvious when just reading the test result. You can find out very easily in post-mortem debugging because you will see the call stack that your unit under test was in when it made the faulty call. And that is useful to understand exactly what happened instead of afterwards seeing

that something happened, but I've lost the history. And as an added bonus, you get a robustness for the entire test suite, because even if you have a crashing bug, you get a full report for all tests. So that's my take on unit testing framework. So you said you get a full stack trace of where the error happened. Is that something you're generating in your mock? No, well, it depends, of course, on your runtime environment. I'm a Linux user, so if I call abort, I get a core dump. From there, I have everything.

Okay. I had seen some tricks recently for generating call, stacks, and C++ when an error occurred, and I was wondering if you were doing something like that, if you were relying on the operating system to generate it. No, no, I don't do anything at all. Okay. I wanted to interrupt this discussion for just a moment to bring you a word from our sponsors. Backtrace is a debugging platform that improves software quality, reliability, and support by bringing deep introspection and automation throughout the software error lifecycle. Spend less time debugging and reduce your mean time to resolution

by using the first and only platform to combine symbolic debugging, error aggregation, and state analysis. At the time of error, Backtrace jumps into action, capturing detailed dumps of application and environmental state. Backtrace then performs automated analysis on process memory and executable code to classify errors and highlight important signals such as heap corruption, malware, and much more. This data is aggregated and archived in a centralized object store, providing your team a single system to investigate errors across your environments. Join industry leaders like Fastly, Message Systems, and AppNexus that use Backtrace to modernize their debugging infrastructure.

It's free to try, minutes to set up, fully featured with no commitment necessary. Check them out at backtrace.io.cppcast. Do you want to tell us a little bit about how you got started writing Trumplay? Was there something you saw lacking in other mocking frameworks, or did you just want to write your own? Yeah, well, it was a combination of frustration and the intellectual challenge. At work, we used, and still use, actually, Google Mock quite a lot. And at that time, this became a problem because we moved our code base to use C++14. And Google Mock at that time did not support R-value references or move-only types.

And suddenly, you had to restrain your design from using the good stuff that C++14 gives because the mocking framework doesn't provide support for it. And that was really frustrating my understanding is that google mock has now moved on they do have support for this and development is progressing so that is not i believe an issue anymore but but it was at the time so i i obviously wanted to solve that and I also felt that Lambdas in particular should make it much easier to use, much easier to offer at least as rich I came up with a design that worked. Internally, it has changed a lot, but not externally. It looks the same. To anyone who is an experienced Google mock user, if you try Tromplay, it will look quite familiar. The inspiration is obvious, colored by my experience. Really simple tests become more or less identical.

But like I said, C++14 with variadic templates, DecoType generic lambdas, that really changed the whole game. So for example, from play, the implementation is a So, for example, TrompeLay, the implementation is a single header file only and the API is small enough that it's... I have a sheet that is written out on two pages and that is pretty much complete for everything in your daily use.

So, you started to touch on the internals of Trompe-l'oeil. Can you want to give us any more details like how it works? How do you generate these mocks, that kind of thing? That's ugly macros. Yeah, really amazingly ugly macros.

And I hate the preprocessor. So that is an inconvenient truth. The thing is, when you want to mock something, you write your struct or class and you say, make mock four, for example. This means that you mock a member function with four parameters. You give it a name and you give it the function signature. And then the macro generates that function. It generates a function with that name, with four parameters of the exact types. If you have

inherited from an interface, you can add an override actually i recommend you do and then there are also a number of hooks in there that are used by other macros to when you place expectations so so you say require call object comma function and parameters and it will then try to find from the generated code which function could be called using these parameters and if it finds one it will create an expectation object and add that to a list of expectations for that member function if none is found you obviously have a compilation error. Or if it matches several, you also obviously have a compilation error. I could go into quite a lot of detail about that

if we had some kind of video blog here instead. But orally, I don't think I can. One thing though, I think probably a lot of listeners find this to be an oxymoron, but the abundance of templates in here makes it possible to catch

many programming mistakes at compile time instead of at runtime, and to provide a very short and informative compilation error to inform you what is wrong and how to fix it. So for example, if you forget a return from an expectation to a non-void function, so you don't get this famous list of 2000 lines of template spew, but you get a message like five lines long saying that, hey, you

really should return something here. How are you doing that? How are you preventing the 2,000 line error, I guess I should say? I recommend you bring on board on this show a German, I think he's a guy named Roland Bock. We've had Roland on.

You had him on, yeah. A long time ago now. He's done a lot of really good presentations on this topic. So some of the ideas I've come up with myself, and many I have stolen from his presentations. Okay. Okay. You mentioned that you're still using Google Mock at NetInsight. Have you tried getting them to switch over to Trumplay?

Yes, it is in use. It is in use at NetInsight. But one thing I discovered was that unless you have a very small test program or very simple test programs, I strongly recommend against translating them. It's not worth the effort. So tests for old existing code

mostly still use Google Mac, but most new written code is written with Tromplay. Okay. And does Tromplay work on all the major compiler platforms? Define major.

It works in Linux GCC 4.9 and later. Clang 3.6, maybe 3.5. I don't remember. I haven't tried the Apple versions. If anyone wants to assist with that and getting

Travis builds for them, I would appreciate it. It works with Visual Studio 2015. I don't know if it works with Visual Studio 2017 because I don't know if it works with Visual Studio 2017 because I've had installation problems. Also there, if anyone wants to

assist with automated tests for Microsoft's compilers, I would appreciate it a lot. I will install 2017 as soon as it comes out, I'm sure, because I need to test my own projects. Maybe I'll give yours a try also when it does. I know you can get the RC now, but I'm sure, because I need to test my own projects, maybe I'll give yours a try also when it does. When the final is released. I know you can get the RC

now, but I'm sure the final one will be out soon. It should be soon, yeah. It's the RC, I cannot install it. There is a bug report on it, and it says problem is solved, pending release, so we'll see. Okay. So, are you looking forward

to reflection potentially coming into the standard it sounds like that's something trump could benefit from very much uh like i said i really hate the preprocessor and it's used to to generate code and it is my understanding. I haven't had time to get my hands dirty and really get into it, but I have built a Reflexper capable Clang that I intend to play with. It is my understanding that with this I should be able to enumerate, for example, pure virtual functions from an interface and generate mocks from that completely within the language proper without touching the preprocessors. So yes, I'm really looking forward to that.

So it sounds like Trumplay requires C++ 14 features. I think you mentioned variadic templates. Is there any other new features you're looking forward to in 17 or 20 that might help out with Trumplay development? One thing that I am looking forward to, it's a small thing, but that is, well, it's not a small thing on its own, but that is structured bindings. Because as it is now with error reporting, if you use your own types, which you often do, obviously,

if you have a stream insertion operator defined, then it will print error messages using that. If you don't have one, you can write your own Trompe-l'oeil print function for your type if you for some reason don't want to implement the stream insertion operator. And I have some automatic support for pair and tuple to print them element-wise and likewise for collections to just traverse them and print. But with structured bindings it will be possible to member-wise print any struct that doesn't have a constructor and that would be pretty cool. It is possible to do now if you're prepared to violate the rules and get an overlay that you type pun, but

if you want to stay within the language rules proper, then C++17 will make that possible. So, that is something I'm looking forward to. Okay. Jason, do you have any other questions? I don't believe I do.

Okay. Is there anything else you wanted to share before we let you go? Yeah, do you have any other questions? I don't believe I do. Okay. Is there anything else you wanted to share before we let you go? Yeah, if you want to learn more, much more, and you have a chance to go to the ACCU conference in Bristol, England in April, I will host a 90-minute session there where I will tell you pretty much all there is to know. Okay. And I saw you have very good documentation on TrumpLay on the GitHub page, and I think

you also had a recording of a user group talk you did where you introduced TrumpLay? Yes, I did. That was when was it? A half year ago, something. It's a very brief introduction. I get all it through the first page of the sheet. But get all of it through the first page of the cheat sheet.

But yeah, it's there. I will make the 90-minute session a lot better. Any highlights you're planning on that 90-minute session that you might want to share now that you haven't already gone over? Yes, matchers, how to write matchers. Say, for example, that you set up an expectation. For example, I have a regular expression matcher that is built in. So if you have a mocked member function that accepts a string, you can say that accept a call to this function with any string that matches this regular expression.

And I will go through how to write your own matches. For example, say that a value must be a member of this set or something. That's interesting. Where can people find you online, Bjorn, to learn more information about you

or about TrompeLay? Okay. On Twitter, my handle is boring. It's my name, bjorn underscore faller. On Slack, cpplang Slack, and on Sweden cpp Slack, I'm known as Rollbear, R-O-L-L, bear as the animal. My GitHub page is also GitHub user is RoelBear. I also have a highly

irregular blog called playfulprogramming.blogspot.se It can be a year between posts. It can be three posts in a week. It depends. Okay. Well, thank you so much for coming on the show today, Bjorn. Oh, the pleasure was mine. Thank you, Roel

and thank you, Jason. Thanks. Thank you. Thanks so much for listening in as we chat about C++. I'd love to hear what you think of the podcast. Please let me know if we're discussing the stuff you're interested in. Or if you have a suggestion for a topic, I'd love to hear about that too. You can email all your thoughts to feedback at cppcast.com. I'd also appreciate if you like CppCast on Facebook and follow CppCast on Twitter. You can also follow me at Rob W. Irving and Jason at Leftkiss on Twitter. And of course, you can find all that info and the show notes on the podcast website at cppcast.com. Theme music for this episode is provided by podcastthemes.com.