CppCast - Factorio

Episode Date: June 14, 2019

Rob and Jason are joined by Michal Kovařík to discuss his work creating the Factorio video game with C++. Michal is 34 years old and started programming when he was 11. C (and C++ soon after) became his favorite language soon afterwards. After quitting University after 2 years he was a regular programmer in a company for 4 years. He then started his own computer game project, which he's been working on for 7 years already. The game is much more successful than anticipated (with more than 1.7 million sales) while still in early access. We are close to finishing the game and deciding what to do next. News C++Now 2019 Videos being uploaded Clear, Functional C++ Docs with Sphinx + Breathe + Doxygen + Cmake Fuzzing Unit Tests with DeepState and Eclipser Michal Kovařík Michal Kovařík's GitHub Links Factorio Sponsors PVS-Studio Facebook PVS-Studio Telegram PVS-Studio Twitter JetBrains Hosts @robwirving @lefticus

Episode 202 of CppCast with guest Michael Kovarik, recorded June 10th, 2019. Sponsor of this episode of CppCast is the PVS Studio team. The team promotes regular usage of static code analysis and the PVS Studio static analysis tool. And by JetBrains, maker of intelligent development tools to simplify your challenging tasks and automate the routine ones. JetBrains is offering a 25 discount for an individual license on the c++ tool of your choice sea lion resharper c++ In this episode, we discuss documentation tools and fuzzing. Then we talk to Michael Kovrig from Woob Software. Michael talks to us about the Vectoria video game that his team is building with C++. Welcome to episode 202 of CppCast, the first podcast for C++ developers by C++ developers.

I'm your host, Rob Irving, joined by my co-host, Jason Turner. Jason, how's it going today? I'm doing all right. Rob, how are you doing? Doing pretty good. Don't have too much news to share myself, you? No, I actually don't think I do either at the moment. That's kind of weird, huh? You always got something going on. Okay, well, we'll jump right to some feedback for the week.

In this time, we just got a tweet from Bigaloo, and he's saying, Listening to Cppp cast has now become a habit makes my train journeys worthwhile so yeah glad you're able to uh you know do something slightly productive during the train ride glad you're enjoying it you know i have had a few people complain to me at conferences that they're like i discovered cbp cast you know eight months ago and i've been listening to an episode every day on my commute. And now I'm out of episodes. Yeah. And now I have to wait. And I'm like, I'm sorry, I'm not sure what to say about that. But maybe this is just a warning. If you're if you're picking up

the podcast right now, maybe pace yourself a little bit. Don't just binge them all because you will run out. We don't have 1000s of episodes or anything or anything i mean you kind of have to bend them a little bit if you just started and we got 200 episodes out there well yeah but i mean you know like do a mild binging like three or four a week not like uh five to seven a week or something like that yeah that's good advice because i don't think we're ever going to be doing more than uh more than one a week no i don't believe that. Okay, well we'd love to hear your thoughts about the show. You can always reach out to us on Facebook, Twitter, or email

us at feedback at cpcast.com. And don't forget to leave us a review on iTunes or subscribe on YouTube. Joining us today is Michael Kovarik. Michael is 34 years old and started programming when he was 11. C and C++ became his favorite languages soon afterwards. After university after two years he was a regular programmer in a company

for four years he then started his own computer game project which he's been working on for seven years already the game is much more successful than originally anticipated with more than 1.7 million sales while still in early access and they are close to finishing the game and deciding what to do next michael welcome to the show welcome hello we'll definitely talk more about the game in a moment here but i'm curious what computer system you started programming on i'm looking at this probably mid 90s is that right it was yeah it was 286 okay sounds familiar yeah yeah and i we called it a blue c i don't know if you know what the borderland i think it was the borderland turbo c or something like that right so that that was the beginning and so that was a 286 dos then i guess probably yeah yeah yeah those and text mode

and 320 by 240 pixels graphics modes etc etc It was a fun time. Do you have a miss? Oh sorry go ahead. We had especially in these times when we started as one of the things why we like programming so much that in the school

we had a rule that we couldn't normally play computer games in the computer laboratory but with the exception that you could play games by people that were created

by people from the school so so when someone made a game he didn't have to compete with the worldwide uh uh how is it competition but we were only competing with game for other people from the school so it was much easier to actually make people play our games and and uh it worked really well it was really fun so we made we made at the time we made several small games and people played it like crazy so it was it was nice times sounds like really i'm sorry i keep interrupting you yeah it sounds like now what did you want to say oh it sounds like a really creative way to make sure that people are you know creating games and and you know what am I trying to say? Like encouraging people to be creative.

That's what it sounds like. Yeah, yeah. So and at the time I already I think I was already doing things in C. I don't know if you wanted to make, initialize a graphics mode, we just have these

sacred three assembler commands that you wrote and then it magically started the graphics basically started working and i remember like i was trying to do a game and and we had to do like from everything i made this simple graphical editor and then and then like small graphics library to draw pixels and lines and then everything and like you had to do everything from very very scratch so it was rough and it was really hard to get anywhere because when you finally get a simple game prototype you are already sick of it and everything took really long but it i think it taught us a lot but do you ever miss the days when you were just talking directly to the hardware like that? Well,

not really that much. I mean, this is not the part that I enjoyed. It was just that obstacle that I have to go through, the pain I have to go through to actually express the things I wanted by the language. So I'm not this kind of guy who enjoys going deep just for the sake

of going deep. I only do it for the benefits, basically. Okay, cool. Well, Michael, we got a few news articles to discuss. Feel free to comment on any of these, and then we'll start talking more about the game you work on that you were just referencing. Okay, so this first article we have is not really an article,

but we've been looking forward to C++ Now 2019 videos being available. And it looks like they're starting to come on. It looks like they've posted six videos so far. Three came out on Friday. And it looks like just three more came out earlier today, Jason. Yeah, at a weird time, midnight in the US, basically. Yeah. I wonder who was posting them.

Yeah, I don't know. But we've made a couple of references to C++ Now. in the US, basically. I wonder who is posting them. Yeah, I don't know. We've made a couple of references to C++ Now. We recently had David Van De Voord and Michael Park on the show, and both of them gave C++ Now talks, so looking forward to those coming online. I haven't actually watched any of these yet.

Have you, Jason? No, I haven't had the chance to yet. I want to see some of these we also had a chris juciak on recently one of his videos is here dependency injection a 25 term for a five cent concept i'm sure there's some good ones here yeah i'm sure they'll keep uploading them day by day yes i'm sure they will okay uh next thing we have is a post from Simon Brand on the Visual C++ blog. And this doesn't really have much to do with Visual Studio at all, but it's clear functional C++ documentation with Sphinx plus Breathe plus Doxygen plus CMake. And I was familiar with Doxygen, but I had not heard of Sphinx or Breathe.

So this is a pretty good article for learning some more about new ways to document your code, new tools to do it. Yeah, I read this and I'm like, oh my goodness, Simon, do you really expect us to do all of this? But the TLDR on this is when you get to the bottom here, if you do all this boilerplate stuff, then you can get your documentation automatically generated on the read the docs website for each commit. And if you have an open source project, that part of it is free. The read the docs thing has free support for open source projects, I believe, right? Yes, yes. So that's the real key here is, yeah, it's a bunch of boilerplate to set up. But once you're done, you've got something that's extensible and automatic. Right.

And I mean, there's definitely some cool things I learned about here. If you think Doxygen docs look a little outdated, then you might want to look into sphinx as an alternative or i say alternative but sphinx as simon says in the article still kind of relies on doxygen to get some of the information out of your code um and the breathe tool is just like a bridge between doxygen and sphinx yes sphinx is a Python documentation generator. Okay, so it can work with Doxygen, but it can also work with other forms of documentation generation. Yeah, I think Doxygen can jump out. I didn't actually see where the bridge happens when I was looking through this, but Doxygen has ways of dumping out like JSON and XML and stuff, so it must be picking that up. Yeah, I think Doxygen, you configure it to output XML, and then Breathe is able to read that and pump it into Sphinx.

Right. Yeah. Okay. Anything else? Do you do much documentation, Michael, or is that less of a concern since you don't have a public API? Well, I actually use Doxygen quite a lot, especially in the early days. And we actually use Doxygen in our our project even in the factorio but to be

honest i use it less and less like basically these days i only generate oxygen documentation when i want to see the think last layouts and sometimes other graphical layouts and for any any other things i kind of realize it doesn't really help me that much because i can either see it in the code or or it's complicated too complicated complicated to have it in code and then we have separated documentation so because you know in the in the very early days when i was like i don't know 15 years old i had this need of like oh there's doxygen tool it can generate documentation and it can give me warnings for anything that I didn't document and then I went through the the trap of trying to document absolutely everything because I want to you know I'm this kind of completionist so I wanted to

document every method every functions and I had these kind of methods like get x and document gets you x and things like that. And it took me some time to realize that it's really not needed. So at this time, I am starting to not see the advantage of using a system like this anymore actually, to be honest. Other than seeing graphs of

classes and graphs of calls and things like that. But maybe you can explain maybe you can change my mind. I don't know. Well, it's interesting, because I've talked to a few people recently who consider, I'll see if I can explain this, how they do, that having documentation in your code isn't a sense of bug. And that is because if you needed documentation, the code could have been written more clearly, like stronger typing, better function names, better parameter names,

then you didn't need the documentation. I don't know if I fully agree with them, but I find it an interesting standpoint. I think I would agree with that if you're talking about some internal project where it's only the developers that would be looking at documentation or looking at code. But if you're making some type of library that you want other people to use then having documentation seems like it's still a good idea yeah well i think that i kind of agree with the idea a little bit but still there are cases where you just need to explain the meaning behind things and you can't just put it into variable or class name because it's just a little bit too complicated

like most of the time or you want to give a note about a variable or method some note that might be interesting for someone trying to understand things but I don't know, I think this is a little bit too strict, like this is an extreme idea as well

I kind of agree that for example a typical example, when you have a block in your function and you put a method, what is this block is doing? Then it's kind of obvious. It's like, let's make it, let's put it into a method and use the command in the block as the name of the method. These kind of cases are

obvious, but then they are, you know, in the real world, you know how it goes. Yeah. So not one extreme or the other. No reason to document every single thing and be willing to document when you need to. Yeah, my experience is that it's really important to document as little as you need,

but you don't really know what's understandable and what's not. So it's a great opportunity when you have a team and someone new comes to the team to listen to them and whenever they ask about something how something works then this is the this is the place where you look at it and you think that this is the place that that needs documentation because like the the feedback is really important because it's not possible for the author of the code to understand what's what needs more documentation what's not or if you look or if you author of the code to understand what's what needs more documentation and what's not or if you look or if you look at the code like five years after writing it and all the way yeah that's the worst one when you read your own code and you have absolutely no

idea why you did that yes yes yeah exactly okay and this next article we have is on the trail bits blog and it's fuzzing unit test with deep state and eclipser and uh this is a pretty technical post and i'll be honest i don't a lot of it seemed to go over my head because i don't know too much about fuzzer tools uh jason do you have a chance to read through this one i did although i have to admit also some of it went over my head because i don't know i don't understand all the technical details about how the fuzzers decide the things that they're going to do but the tldr is they have a a new fuzzer called eclipser and they've now integrated into their testing tool, which is called DeepState. And they found several examples where this Eclipser fuzzer is, you know, far superior than some of the other ones that they

already have integrated with DeepState. Yeah, and I think that a good takeaway is right from the bottom. They say like, Eclipser is not going to remove the need for other fuzzers. Every fuzzer does something different. So they say, like, you should be testing your code against all of the fuzzers. Right. Because that's how you're going to actually find the bugs that matter, which is just crazy. Like, I didn't even realize there were, like,

I knew there was, like, two or three fuzzers. I've used them. I didn't know there's, like, six fuzzers. And I didn't know that this deep state thing had them all automatically integrated in with i think it's was it g test or boost test sorry i missed that bit here i forgot i think they compare deep state to google test oh okay well i mean it's got the same kind of syntax that's what it does yeah and and it can just kind of automagically do these things. Anyone who's interested in fuzzing automated code testing really

needs to read this article and check out DeepState. That's the real thing anyway. I would actually like to know I would like to learn what does it even mean? Because I don't know what's fuzzing. I never actually encountered it. That's a great question. I don't know when we last talked

about it on the show. So with fuzzing if you have um an input that can take any kind of data it doesn't matter what kind of data it is you want to run it over all the other all of the possible types of inputs so i've done this with chai script with my scripting engine i can um you can use a fuzzer to generate a blob of data you pass it to the scripting engine, and then it just tries to execute it. And the fuzzers use code instrumentation with like code coverage metrics to determine what functions have been called with the input that you gave, and it starts breeding input files against each other, and all the while trying to execute more and more of your program based on its code coverage output. So it's basically like learning what to give to the program to execute all the paths automatically?

It is, yes. That sounds cool. It's really cool. And the key is to enable it with something like Volgrind, which would slow it down an awful lot. But better is address sanitizer with Clang. So if at any point the input that it gave you were to generate some sort of memory error,

you get an immediate crash with a stack trace, and you know what happened. And now you have a bug that you have to go fix because you have a potential security flaw in your code. I will tell my students, I did all of the things right. I was using all of the unit testing, all of the warnings on all of the compilers, all the static analyzers.

And I knew that ChaiScript's scripting engine was perfect. And then I learned about fuzzing. And in the very first test, I crashed my parser because I never tested handling an input length of zero. And then after a week or two of running with fuzzers, I decided ultimately I had to rewrite the parser because it was fundamentally broken. Oh, wow. Oh, yeah.

We should find someone who could do a whole episode on fuzzing, because it's something we've referenced a few times, but we've definitely never gone in-depth on it. Oh, yeah, and I believe we have contacts who can get us in contact with the people who implement these fuzzers and stuff, too. Yeah. Okay. Definitely an interesting thing that I have to study now after the talk.

Yeah, sometimes when i'm teaching this in a class i have my students all stop and kind of look at each other and like we have a problem we have to go do this right now okay so uh michael we've already made a couple references but you want to start off by just telling us a little bit about your game well uh the game actually it's kind of interesting because not only the programming the game, but also the game itself reminds a lot of concepts in programming, actually. Okay.

Because the game, it's basically a game about automation, which is also programming is also about automation. But it's a game about for even for non-programmers or people that not that uh don't even want to do programming at any time in their life but basically the game like the game is about a guy who creates on an alien planet and he he's there alone and he needs to survive and his only tools are basically some in the beginning he has to do everything manually and then he can make simple machines and then use these machines to make products for him and then he starts to build eventually a small factory

and then he starts to build a big factory but since it's just one player he everything needs to be automated and if he wants to expand expand more and more and in a bigger scales and make more different products he need the parts of the factory to be more and more autonomous so anything you do you need to think with the with the thing in your head that you are only one guy and need to really think about your time and how to invest it because almost everything can be automated at some point but you need to really think about your time and how to invest it because almost everything can be automated at some point but you need to really think about what has the best like how can you what what saves you most time with least effort basically and this is kind of the similar what we do when we do programming or anything related to computers or even in real life like where's the threshold like

can i i don't know do i spend half an hour doing a script that saves me five minutes a week or not you know things like that basically so and we try to and then we do this idea and then we try to wrap it into a game that would be actually fun and we were really curious in the beginning whether something like that could be actually fun for people because we enjoyed it a lot but we had no idea if actually enough of people would enjoy that and luckily enough actually it worked because i mean obviously it's not a mainstream game it never will be but it's way more mainstream than we could ever anticipate because we didn't try to make it like hard like we didn't try to make it

look elite or something like that like you you can understand you can play the game from beginning you can understand the concepts the basic concepts are super simple and uh if you want to play the game in a simple way you can finish the game in quite simple way and if you want to play it complicated if you like want to make complicated system you still can do it so people kind of enjoy it so uh yeah you told it it's like it's all like in the beginning we were like oh if 10 000 people buy it it'll be super crazy and now we we are here with 20 people on the team so it's uh it's we're doing it for seven years so it's kind of uh the how to say the definition of when it's

finished is changing all the time because in the beginning it was supposed to be finished in half a year it was half a year project and now it's seven years but it's there are many aspects of this and mainly like you you know like you finish you you go one iteration through the game and then you're like uh your standards gets higher and then then you do second through iteration through the game and then you're like your standards gets higher and then then you do second iteration through iteration then the game grows and then everything like the the kind of a problem of the game is that it has like many aspects it combines many different games like there's logistics which means like basically you have big many parts of for example transport icons include included then you have like technology and some kind of combat so we have parts of for example transport icons include included then you have like technology

and some kind of combat so we have parts of parts of real-time strategy or civilization of this kind of gameplay included and then you have crafting and so it's like some partially like some industrial games and and many others many other small things in between So basically there are a lot of different systems in the game and as they grow and as each of the systems are improved each of the systems are improved over time it takes more and more time to improve all the systems at the same time and they all are interconnected. So it's starting to be quite a big project.

And I think that I'm saying that when when you see a game you can very very often you can clearly see especially if it's in an indie game if the game was started by a programmer or by a graphics graphic guy and here on this game you can clearly see that it was started by a program if you see all the system they're moving and everything going on. You said it's kind of like programming, but not really like programming. What does this look like? How does the game player automate things and program the system, I guess? Well, for example, I don't know.

The first thing that you encounter is that you, let's say you mine coal and you mine it by hand and then you make a first machine that runs on coal and can mine and for one piece of coal it can mine, I don't know, seven pieces of coal and

then the first automation is that it that you put the output of the machine back to the input of the machine, and you put two machines next to each other so they feed each other by coal. And suddenly, like, the two miners, like, you know what I mean. Like, the output of one miner is put in as fuel into the other miner. Right. And from the second mine to the first one.

And suddenly, you look at it, and you made your first your first like little super simple automation and and it continues like that then you have transport belts so you start like in the beginning you like you like hoard on one place you hoard iron on second place you hoard coal and then you start using using transport belts and you move these two things to the same place and automatically put them in the furnace instead of doing it manually and then and then you play for some and then you always take the finished i don't know iron plates from the furnace and do whatever with them build some machines and then after some time you learn that it's better to put the iron away from it and build the products you do manually automatically and from the iron plates you make i don't know another transport belt and other inserters. Basically, in the beginning,

the main products of the factory are other pieces of the factory, so it grows, feeds itself. Wow. And basically, the programming part is that, for example, you always, like, the typical experience of people is that they start building the factory

and after two hours they realize that it's a horrible mess. They can't extend it in any way, and they had to start over and think like, okay, now I will build it properly from the beginning. I will leave space for extension of all the system in the factory so I can expand and scale it up. And after five hours, I figured out that I didn't prepare enough, and I just start again. This is the typical story, which sounds very familiar when you think about it.

And by iteration, you learn how to scale things up, how to prepare for how to divide individual systems, and how this all, I think, is similar to programming. And there's obviously much more. And the more complicated the factory is, the more you need to prepare before you start just building, doing the buildings. And obviously, even if it's a shitty factory,

even if it's a messy system, you can still finish the game. It's just going to be a horrible experience. That really does sound a lot like programming. Yeah. So I want to dig more into the technical aspects in a minute, but I'm just curious. You said you can finish the game fairly quickly, or you can take

a long time building it for factory. What is the end goal? You said you're an astronaut that crashed on this planet. Are you trying to build a new spaceship to get off? Yeah, your goal is to launch a rocket into space. Okay, okay. And for some people, it's just the beginning because launching the rockets

allows you to do infinite research and then people just make huge factories for, you know, then they basically, for some people, it's the end of the game but some people they call it rackets per minute like how many rackets per minute can you launch and then people like put their crazy numbers like I put I don't

know I launched 100 rackets per minute and someone like no this you need to push these numbers you need to launch 1000 rackets per minute etc so it's uh basically like it we really wanted to have it to make end to the game because when it's the game

still like you have no no goal from the beginning it's kind of weird it's like too open for my for my test taste but even if you finish very often you just want to finish and scale it bigger and bigger. Okay. And it sounds like you can get the game now in early access, but does the game have a final release date planned? Yes. Well, we are, I think for like three years, we are saying that it's going to be this summer or the next summer. But I think we are now really like trying to finish it but basically I mean the game is in early access but I would say it's quite

stable because we have this stable and experimental release cycle and the stable releases are kind of I would say very stable because it's played by a lot of people so and we try to fix as many bugs as we can

so the game is stable the game can be finished and it contains almost everything that we wanted for the full game basically now we are polishing stuff like making more understandable uis and making better tutorials and finishing the graphics and everything but I think that it's not like a half-finished game at this point. So I'm curious about this timeline, because you said that you originally thought the game would be done in six months, I believe is what you said. Yes. How long did it take you until you actually had something released

on early access from the time that you started? Well, if you mean early access on Steam? Yeah, or whatever. From the time that you started to the time that you were actually selling it. The thing was that, yeah, I think we took something around a year and a half before we started a crowdfunding campaign.

Okay. Basically, me and my friend, we saved some money and moved together to an apartment. We were working like 14 hours a day, seven days a week. Okay. We had both some experience from companies and did a lot of projects, etc. So we were working on this. And after one and a half years, the money was starting to dry out. And so we needed to... And also, we weren't sure if, as I said in the beginning, we weren't sure if actually someone would enjoy it,

if it's not just us who liked the idea. So we needed the money and we needed reality check so this is why we made the indiegogo campaign and we made you have like these two kinds of campaigns you can either have like whatever money you get uh you get the money and you do the best you can with the money or you can have like a strictly specified minimum goal and if you don't reach it you you don't get any money. And the second sounds like it's not as good as the first one because you risk that you get nothing,

but we preferred that because if you have some estimation of a budget and you get half of it, what do you do with it? Right. Yeah, so it would be the worst. So we specified a goal that at that time we thought it would be enough to finish the game. I think it was like 17,000 euro. Okay.

Which sounds funny, but... Yeah, not a lot, yeah. Yeah, it's not a lot, but we were living, two guys in an apartment in Czech Republic, eating cheap food and like not really having many expenses and it was it was really like we really thought that we will finish it soon so yeah we've and eventually like the campaign was it was it's a story for of its own but eventually we like we passed the campaign uh at the end and we thought that everything is going to be fine but so it was

after one one and a half year but uh we still like basically after campaign the the like any people didn't buy it anymore like they bought it they were buying like we during the campaign we made a web page and there we started selling the page the game directly on our web page but two days after the campaigns, the hype basically died out. And we saw that we can finish the game. We hired one of our friends, so we are like three programmers and one graphic, and

suddenly the sales went to zero for several months. So it was a tough time, actually. And then in this time, we were mainly working on the backbones of the game, but then we realized that the graphics is important for selling it, so then we started improving the graphics more and realized what's wrong. And slowly, slowly, step by step, the sales on our page were increasing increasing i think after

like two and a half years we get got to a stage where we could like live very like we could have a very humble life paid from from the game really humble but yeah something something around that and then it was and and basically we were we we didn't want to go to Steam and it's too early because from what we made some research and we found out that the first release on Steam whether it's early access

or not, the first time you appear on Steam is the most critical for the game. And you really want to be ready for that. And if you invest like half a year extra to polish the game and you really want to be ready for that and if you invest like half a year extra to polish the game more and do the release later it's really usually worth the trouble

so we actually waited two more years for the Steam Early Access to really use this opportunity as much as we could and it really paid off a lot because I think that in the first week of being on Steam,

we sold as many copies as two and a half years before Combined. It really pays off to do the first impression properly. That was our takeaway. Sounds like a really important lesson. Yes. Actually, there are articles about it, basically. I don't know if you know about Steam Spy.

It's because the problem with Steam and the sales this is more about games than C++. It's not public how many copies of the games are sold and how is the you know what is the graph of the sales and etc but some guy discovered that uh people like

share what games do they do they buy on their profiles and basically what he did he made a robot that went through individual profiles of gamers basically random profiles of gamers and he he made some some statistics that if he discovered like 0.1 percent of gamers, basically random profiles of gamers, and he made some statistics that if he discovered 0.1% of gamers, he could make a rough estimate of what are the sales overall. So basically, this unofficial page is making

statistics of sales, which is kind of interesting, and then there are interesting articles about takeaways for indie devs and basically he discovered that people thought that early access is like one big

chance to make big sales and then people thought that when they go from early access to full game that it's even bigger or it's like second chance to make the same kind of impression and from the statistics it looks like it isn't like that. When you go

from a relaxes to full game, you have some kind of bump. It helps, but it's nothing compared to showing the game for the first time. So, it's just. I want to interrupt the discussion for just a moment to bring you a word from our sponsors. PVS Studio is a tool for detecting bugs and security

weaknesses in the source code of programs written in C, C++, C Sharp, and Java. It works under 64-bit systems in Windows, Linux, and macOS environments, and can analyze source code intended for 32-bit, 64-bit, and embedded ARM platforms. The PVS Studio team writes a large number of articles on the analysis of well-known open source projects. These articles can be a great source of inspiration to improve your programming practices. By studying the error patterns, you can improve your company's coding standards, as well as adopt good programming practices which protect from typical errors. For example, you can stop being greedy on parentheses when writing complex expressions involving ternary operators. Subscribe to Facebook, Telegram, or Twitter to be informed about all publications by the PVS Studio team. Links are given in the technical stuff a little bit more. Are you using any existing engine or graphics libraries that we would know about?

Or did you kind of build all that stuff yourself? Okay. As I was talking about those times from the beginning i don't know if you remember there's this uh library called allegro i don't know did you hear about it i have heard of it yeah yeah it's it's it's it basically has roots in the old times like when in the dos era basically and somehow when i when i started doing factorio it was the i checked if it still exists and it still exists existed so basically all i basically. And somehow when I started doing Factorio, I checked if it still exists, and it still

existed. So basically all I wanted from an engine was the Drome Sprite and player sound. That's all I wanted. I didn't want to use any... I made a really fast

test of what engines are available available available and the problem with factory is that i wanted i from the beginning was super clear that we will need a really good performance because the game is based on automation and and that and the interesting parts of it emerge only when you have begun a factory that's the time where it starts to be interesting because you need to manage all the mess so and i needed all the factory to be running so it was very obvious from the beginning that it will have to be optimized a lot and i was afraid that if i used an engine on the market i i would i would basically at some time i would hit a wall and there would be no way for me to overcome it.

So I wanted to have everything under control. This is why we use Allegro basically just to draw a sprite and play a song and read from keyboard. And everything other than this was written by us. And actually, a year ago, we actually moved from Allegro to SDL. But it's not really like... It's basically just a big... It's not that big part of the code being changed because most of it is the game logic.

Right. So both of those pretty much just give you a place to draw some pixels and some I.O., basically. Yes, yes, basically. And actually, we spent a lot of time with optimizations while doing the game. Basically, the cycle was that once the game started to be public, people made as big factories as they could, and then they complained on the forums that it's starting to slow down.

So we were like, okay, so let me look at it. And then we've optimized the game with the save as a reference, and we optimized it enough to make it like, for example, twice as fast, so it's smooth again. And we put this into the next version.

And the next version, people realized that they could build a bigger factory, so they made suddenly two times bigger factory and then they complained again and like this basically we went through several cycles like this and the good thing about this is that suddenly we could optimize

with a reference to real factories and not against some theoretical things so I think it's like there were many steps like that when we made it two times faster and with C++ it's really like this is where I was happy that we chose this way

because we could do really interesting things and obviously like the further we got the more sophisticated tricks we had to go like the low hanging fruits are kind of depleted so yes so it's like this so this game's been in development for a while now that you've said

you've you've watched several uh new releases of c++ basically come in i mean new c++ standards have you what standard do you use have you morphed what compilers you require and what standards you use along the way? Basically, I remember the time when we adopted C++ 2011, or C++ 11, and we were really happy with that because one of the problems is that

we do Linux, Unix, and Windows releases. So whenever we want to use something, it needs to be supported by all of the compilers that we use, which kind of slows things down. But for example, in the beginning, we used Boost. And I still remember the moment when we removed Boost for each by the C++11 for each, which improved compile times by like 10 person or

something so yeah it was really crazy and for example another thing that we used boost at these times and i think since c++ 14 or something we got rid of boost eventually completely okay and the main the main motivation was compile times because And we will probably get, we can talk more about it, but basically my biggest complaint about C++ overall, it's compile time. It's basically, it's tightly related to the include system. Include system and compile time, they are like the worst cancers of C++.

And like, I think I wrote somewhere that like if you include like string or something you include like or Iostreams or something you include like millions of lines of code in single include and I remember I was doing tests like doing 500

CPP modules that were completely empty and just included one string or something like that even with the pre-compiled headers it started to compile like half a minute or even more so and when you see this you start to be desperate

because we were like we were trying to optimize our include structure forward declare everything and etc. But if you know that just including string makes your compilation that slower,

that's kind of desperate. So over the years, like when we started using C++11, it started to be better because since we got rid of Boost, it was better. And then basically once a year, someone was too annoyed by the compilation times,

and they tried to make something. For example, the last time when we were doing it, we discovered FastBuild. I don't know if you know that. You know FastBuild? Yes. Without that, it would be really horrible. Basically, when we adopted fast build they include

compile time improved basically for the people who don't know what fast build is doing it's basically doing it's trying to solve the compile time by putting things into modules like if you have 1000 modules they group them

into modules with the ideal count because if you group I don't know, every 10 CPP files into one, it means that the preprocessor doesn't have to process the include files again and again in every module. So if you put and then it makes some other optimizations. And basically with fast build and a lot of hacks, we are now, I think, the recompile time on debug is something like two to three minutes, which is

not that bad for a whole project like this. But still, every second you have to wait, every time when you compile, it's adding drag to development. And basically this means that as the project gets bigger it's the development gets slower not only because it's inherently more complicated but also because of this and i think that like sitting there and just waiting for it

to compile it's it's it's really horrible so this is our biggest biggest fight i think with c++ so you are you currently on C++ 14 then? Is that what you said? I think we are using... To be honest, I'm not really actually sure. Okay. If we are using 14 or 17. I think usually it works in a way that

we are trying to use the latest compilers. And I think that this kind of... On which C++ version you are, it's kind of bending now because yes there are like standards right of what what what's this c++ 17 but in reality it's about what is supported by the compilers so usually it works in a way that someone's like oh there's this cool c++ feature let's test if we can use it and we make a branch and we try to use it and see if all the components, because we have automated testing of branches, et cetera.

So we test if all the components can use it. And if so, then we'll say, okay, so we can use this C++ feature. And so this is usually mainly the reason why I can't tell. That makes sense. And for our listeners, because you didn't explicitly state this yet, you support all the way back to Windows XP,

Ubuntu 14.04, is that right? According to your GOG.com page? I think that we dropped XP already. Okay. Because we dropped 32 bits like a year ago or something. That's fair, I think, yeah. Yeah, because it was too annoying to solve all the problems.

Because we have several constraints, and one of the constraints is that our multiplayer is based on deterministic lockstep, which means that the logic of the game needs to be fully deterministic, which means that any small change in behavior on different machines in the multiple causes desynchronization and it just can't happen,

which means that a lot of things can get broken, basically. Like, I don't know, you make a steady set of something based on pointers, which is normally something you could do easily, but here you can't because the order would be different on different computers. Like many different things can get broken. And with 32-bit versus 64-bits, there are a lot of small details where you need to be careful. And basically this is why.

And I think that based on the research, less than 1% of people use the 2-bit system. So it's not supported anymore. One thing I'm curious about is I saw on your forums that you allow mods of the game. Did I read that right? Yes, yes, yes. Well, that's one of the reasons why I wanted to support mods

from the very beginning was that basically one of the reasons why I wanted to support mods from the very beginning was that basically I one of the games I came from was the Minecraft modding community I mean I wasn't modder but I was playing with mods basically and and when I looked I don't know how it is now to be honest because I didn't have time to play the game for years but before as some the modding it was really wild because for example i don't know the game didn't have like they had like uh numerical constants for things like for example you had ores and they had to make like consortium of mods where they made a spreadsheet of what ids of different ores used what mods so they don't have conflicts and things like that so like the people really had to cooperate outside the game to

make things compatible and that looks like not really a good system and also basically I like to mod modding in most games like in Baldur's Gate modding is really great but also it's this for a

user it's really horrible experience because you have these pages and pages of uh of scripts of how how the modding needs to be done and what what is the order of the files and all the special operations and once you start playing with the modded game you can't ever change anything because it gets broken so basically we put we put we put the constraints on the game that the modding should work in a way that you can, at any time, you can add or remove mods. When you have a save, you can add or remove mods.

And it needs to just work. And when some objects from the mods are not available anymore, they just disappear from the save but you can continue and and that also that the mods obviously shouldn't be able to break the game in a way that you don't understand what broke it which is the second most typical problem like for example in oblivion modding where typical thing was that you just put more and more mods and suddenly this game started crashing and then happy hunting like what do you do like and when you change the mods like if you want to for example

bisect the mod list and to find out which mod is causing it you can't really do it because when you remove the mod you break the save game but to get the save game to the state where it starts crashing again takes you like 50 hours of gameplay so virtually impossible to find out what's wrong and

this was such a so you can see i had so so much frustrating experience with mods that i wanted to make it right which is adding another huge constraint because you need to deterministic game and you need two mods like this where you need to when you need to be able to remove and add mods at any time during the like not during while playing real time but but during one playthrough basically so and the mods yeah and also the mods are one of the things that i think was a mistake is that we use the lua language for mods and from my experience with lua i have based on basic i

didn't have experience before and i'm not really happy with that because Lua has basically three fundamental problems. One is that in Lua, you can't really deserialize and deserialize the state of the state. So basically, you have a scripting language, but the scripters, they need to work properly to not desynchronize the game in multiplayer, which is really weird because if they do it, it's like...

I mean, we have a guide for them, which is not that hard to follow, but still it's really like something... Good scripting language should be able to serialize its state completely, right? The second problem with Lua is that it indexes things from one not from zero it's like if i could count all the places in the in the lua to c++ transition but it's like plus

one or minus one base to make it compatible with lua it's like really crazy and the third and the third thing biggest problem is that in lua like everything is global by default like every variable you need to explicitly say it's local instead of instead of otherwise which i think is really weird like you make this one little method use this variable and you don't use the local when when creating the variable then suddenly it's global it's kind of strange so but we didn't realize all these problems so we just i'll kind of living with what other hand a lot of people are used to that and they are

but if I was to make another game I would definitely try to do something else either some other language I don't know maybe if there is some good script basically the best scripting language would be if it was strictly type C++ for me at least buting language would be if it was strictly typed, C++,

for me at least, but I don't know if it even exists, actually. Well, Jason might have a scripting language to recommend for you. Well, ChaiScript... It's the ChaiScript language. Yeah, ChaiScript is strongly typed, and it is one-to-one typing with C++. Anything you expose

in ChaiScript is a C++ type, but you probably would not be satisfied with it because it would increase your compile times and it is pretty heavy on dynamic allocations. I think that I saw... Didn't you have a talk about optimizing ChaiScript? I did, yes. I think I saw the talk where you...

Now it's starting to connect in my head so maybe I could still maybe it could be interesting to I should check it out probably sure check it out yeah so basically yeah this is about the Lua but for example one of the great things that I

like I think it's really for me to understand like what were the good decisions and what were the bad decisions. One of the good decisions is that in the morning you can define your definitions in Lua, but the good thing is that the definitions

of the objects are also programming language itself. It's processing a script that in the end generates the tables that are then used as input for the game of the objects, of definitions of the game objects. And since it's a programming language, it gives you huge power. For example, you can have a mod that modifies data of other mods, and it can be done procedurally. So these kind of things are really, really nice.

Did you use any Lua bind or Sol or anything like that to integrate it with your system? Or are you doing all that binding by hand with Lua? Well, we have... I'm just looking at the source code at the moment. We have Lua. I think I would say that we do it ourselves, probably. We have some classes

that do it, but it's written by us. Maybe we started with some library, but it was written so many times that basically we even have our own version of Lua because Lua isn't deterministic actually. And we had to rewrite parts of Lua that

when you iterate, I think, tables or something like that, to make it deterministic. Wow. And also, yeah, because we just need deterministic simulation. And also, Lua was not deterministic on the same systems. Like, for example, if you, like, in Lua print minus, I don't know, if you print minus zero, it prints different values on different platforms or like basically all the

print functions give you different results on different platforms because the print has different implementation on different platforms. And so we had to do several things like several, or for example, the random function can be just a random function, obviously if you want deterministic simulation so it needs to be

but and also the problem is that basically we need to simulate like objects in Lua because we are to the user we are pretending that you have like structured data and objects but

it's not really like that in Lua so we are doing some weird dark magic to to to try to to make it look like that and it's and the problem is that for example in the beginning we were doing things like i don't know you expose you expose a class and we are like pushing all the possible i don't know methods and date methods and access points of the class and the problem is that then again you have the optimization constraint and if we found out that it's slowing the game

too much, for example now we have to pre-prepare access tables for each object and then reuse them and things like that. So there were also many iterations in the Lua API that we had to change and optimize.

So basically, if I wanted to use scripting language, again, in a project that's so constrained by optimization, you definitely would have to have some better access to structured data that's really fast.

Something that you said sounds really interesting like did you expose somehow c++ but you can't really expose c++ because you can't let the user do whatever you still need to go through some api but i don't know something that's more like yeah structured yeah that is part of the problem i mean and it's a trade-off if depending on how much power you give the user, then you're giving them too much power into your system. You know, like how much do you need to sandbox it or whatever it sounds like

is also something you have to be concerned about. Well, yeah, when it comes to modding, it's very clear. It needs to be the user shouldn't be able to, not shouldn't be, but can't be a threat to the safety of the application.

That's like number one rule. And it shouldn't be able to crash the game or in unintended ways. So basically, if you just give him an object and let him do whatever with it, then basically you would have to check that all the values of the objects are correct. Or you need to just allow him to change any value

through your API. I don't see any way around it because as I've learned, it's better to sacrifice a little bit of performance than have broken mods. Right. That makes sense. Yeah.

Okay. Well, it's been great talking to you about all the work you've made on the game today, Michael. Maybe just one more question before we let you go. You talked a little bit about your criticisms of C++ with includes and compile times. Is there anything you would like to see added to C++ that might make it better? Well, when it comes to includes, there is this, I think it's called modules, right? Right. There is this module thing.

Then for me, it would be the more important things that anything combined that's planned. And I know that it's in standard and it's, I think, experimentally supported by Visual Studio. I made some experiments and it is experimentally supported, but it's still way slower than even includes. So it's not like, it's not, I guess it's just, it's not in the phase where it would actually make things faster. Right. Hopefully that support will get better when it's more of an official feature in C++20.

Yes. Yes, I hope so. I'm not really sure how it is, like all the standard libraries would probably have to be rewritten to use modules. And basically all the projects that want to use it would have to be rewritten in order to make it actually to provide the advantage of the faster compile times. But I would be really happy if it solved the compile time problems because for me it's one of the biggest burdens.

And for Xchalade, the thing I'm now kind of watching is the Jonathan Blown's language. I probably heard of it, the Chai. Did you hear about it? Jonathan Blown's? I've heard about it, but I haven't looked at it myself. Yes. Yeah.

I watch several streams, and it looks like he has really interesting insights and basically the problem is i i kind of see the problem even with the way the question because they are questions like what would you add to c++ and i think the biggest problem is that we are still trying to be backwards compatible and they are adding more and more and more things on top of each other and on top of and then when someone wants to learn c++ it's super hard like i i'm programming in c++ for more than 20 years and i i'm quite sure that there are there are a huge amount of things that i don't even know that they exist or I wouldn't understand what's going on because they are too obscure or the language is just too complex, I would say.

And it's not really necessary. It's just because we had to solve this problem of making compatible and basing it on language that's 30 years old. And I think it would really be interesting to either try to adopt some new language or maybe make C++ 2 or something where people would just cut off

the garbage and just make it backwards incompatible and just try to move on because this problem of always finding the local maximum, like improving the local things and just

trying to get it a little bit better has its limits and I think we are reaching them. I kind of believe that someone needs to be brave enough to try some bigger step. That's my feeling. But other than that, obviously the modules, if you want a simple

answer. I don't think you ever actually said the name of this game. The game is Factorio. Okay. It's like Factory and Factorio. It's like a Spanish kind of version

of the word. Factorio.com is our webpage. Okay, great. Well, it's been great having you on the show today, Michael. Thank you. Thank you for inviting me. Have a nice day. You too. Thanks so much for listening in as we chat about C++. We'd love to

hear what you think of the podcast. Please let us know if we're discussing the stuff you're interested in or if you have a suggestion for a topic. We'd love to hear about that too. You can email all your thoughts to feedback at cppcast.com. We'd also appreciate if you can like CppCast on Facebook and follow CppCast on Twitter. You can also follow me at Rob W. Irving

and Jason at Lefticus on Twitter. We'd also like to thank all our patrons who help support the show through Patreon. If you'd like to support us on Patreon, you can do so at patreon.com slash cppcast. And of course, you can find all that info and the show notes on the podcast website at cppcast.com. Theme music for this episode was provided by podcastthemes.com.