CppCast - Rigel Engine

Episode Date: April 9, 2021

Rob and Jason are joined by Nikolai Wuttke. They first discuss a blog post series from Raymond Chen on coroutines and the upcoming pure virtual C++ conference. Then they talk to Nikolai Wuttke about Rigel Engine, a modern C++ reimplementation of Duke Nukem II. News C++23: -> and :: to be replaced by . operator C++ coroutines: The mental model for coroutine promises Mutabah's Rust Compiler Pure Virtual C++ 2021 Conference Links Rigel Engine on GitHub Play Rigel Engine online Sponsors PVS-Studio. Write #cppcast in the message field on the download page and get one month license Date Processing Attracts Bugs or 77 Defects in Qt 6 COVID-19 Research and Uninitialized Variables

Episode 294 of CppCast with guest Nikolai Vutka recorded April 7th, 2021. Sponsor of this episode of CppCast is the PVS Studio team. The team promotes regular usage of static code analysis and the PVS Studio static analysis tool. In this episode, we discuss coroutines and virtual conferences. Then we talk to Nikolai Vutka. Nikolai talks to us about Rigel Engine,

a modern reimplementation of Duke Nukem 2. Welcome to episode 294 of CppCast, the first podcast for C++ developers by C++ developers. I'm your host, Rob Irving, joined by my co-host, Jason Turner. Jason, how are you doing today? Okay, Rob, how are you doing? Doing fine. You've been kind of giving us updates about your your laptop as of late has the saga come to a resolution i sure as heck hope so i've done some work with i just received a new laptop i managed to find it on lenovo's clearance site i think i

told the group that one that i'd ordered it on the last chat um and i've done a few things with it and it seems to be working very nicely. So it's nice to have something with a real GPU in it and stuff. It's always nice setting up a new laptop. Do you do anything to kind of automate the setup of a new machine? No, I don't do it that often. And, well, I do do a lot of work from VMs, but even so, just setting up Windows and installing Visual Studio, you know, like that doesn't take that long.

To me, it hasn't been worth it. But I have worked on those automation scripts before for setting up CI build machines. I just haven't ever seen a need to do it for my own personal systems. Yeah, you're absolutely right. It doesn't take that much time. You don't have to do it that often. But I definitely have in the past created like a chocolatey script to install a bunch of tools that i knew i was going to want and then i probably forgot about it the next time i installed a new machine

i always feel like like whenever i set up a new system like i'm going to use chocolatey for managing all my software this time and i'll install like two things with it and then i just go back to managing it manually like so it's not a part of the operating system, I guess. You know, I would like to make a quick comment here for long-time listeners that in our notes that we look at every single week, there is a note here still that says, is Jason not clicking his pen back from PenGate?

That was like probably 230 episodes ago now. Yeah, we've gotten pretty good about uh trying to reduce the extraneous noises in the show but we still uh still have to remind ourselves okay well at the top of every episode like a piece of feedback uh this week we got a tweet from patrick and this is in reference to uh last week's episode with David Barr. I found out about this today, referring to the Pixel game engine. Thanks to CppCast and kind of want to see what it would take to adapt to Android. I've been dying to find a pure C++ engine that's easy enough for beginners that I could use it in my videos. So that's pretty cool. I hope he's able to uh have some success with that and get the

game engine working on android because i think i mean david talked about how it's already on linux windows web assembly i don't think we talked about any mobile platforms though right i don't think so yeah yeah but i mean it's already pretty darn portable so and it's it's what he said basically open gl one or two or something ridiculously old is what uh what it supports by default so it's got to be a path for it right and some i think you mentioned that some people were trying to get it working on like vulcan and then some of the newer uh you know not game engines but newer uh ways of rendering i don't know the way to describe that.

Yeah, Vulkan was, I think, yeah, right. All right, well, we'd love to hear your thoughts about the show. You can always reach out to us on Facebook, Twitter, or email us at feedback at cpcast.com. And don't forget to leave us a review on iTunes or subscribe on YouTube. Joining us today is Nikolaj Vutka. Nikolaj is a software engineer with a

strong interest in modern C++, reverse engineering, graphics programming, and game development. He works as a technical principal at Ableton, a Berlin-based music tech company. In his free time, he plays video games on the guitar, works on Rigel Engine, and contributes to other gaming-related open-source projects. Nikolai, welcome to the show. Hey, thanks. Welcome. We're planning to spend most of this interview talking about your game engine because that's the cool, fun thing. But can you tell us a little bit about what you do at Ableton? Sure. So I work on a team which works on a product called Push, which is kind of a hardware controller for our main software product. So we have a digital audio workstation software called Live. So yeah, that's kind of full-on music production,

music editing suite. And it has this hardware controller that you can use to, on one hand, it has these pads that you can hit to play drums or also play melodies. And it has a bunch of knobs that you can use to, on one hand, it has these pads that you can hit to play drums or also play melodies. And it has a bunch of knobs that you can turn to get a bit more of a tactile feel. And yeah, because often when you're tweaking a virtual synthesizer or something, it feels better if you can actually turn a knob instead of dragging the mouse. It feels more natural to do it this way. Right. can actually turn a knob instead of dragging the mouse it feels more natural to do it this way right and yeah i'm on the team that develops the software for this hardware product interestingly

the hardware itself so we don't really do much um like embedded or low-level um firmware coding for the hardware itself um because most of the logic actually happens on your computer. So you connect this thing to your computer and then it controls our software. And it has all these buttons with LEDs in it so they can light up in different colors and stuff like this. But all this control messages of how to light up these buttons are generated on the computer. And so the hardware is pretty simple.

It's pretty much just sending data about these button presses to the computer and receiving information about how to light them up. And also has a little display, which shows some stuff on. And so yeah, we write all the software for that stuff. Which interestingly is mostly written in Python. So we have some kind of embedded Python interpreter deal going on. And so then there's a lot of infrastructure. So the main software is written in C++. And then there's some infrastructure that kind of runs the embedded Python code. So I was curious, and now I feel like I might already know the answer, but how real-time

does this need to feel? If you're running Python embedded, I know it can still be very fast, but if you've got a little bit of lag, does the user notice or what? This is actually a pretty big part of this thing to make the things that need to be real-time actually real-time And so we kind of have a distinction between non real time inputs, like if you want to create add some kind of new effect or something, that's not something you do in real time. So that's fine to handle that in Python. But something like playing a note or playing a drum actually needs to be real time. And so this is then handled directly in C++,

and the Python code is more like orchestrating this, and it's saying, okay, from now on, I want the messages from the hardware to be going directly into the C++-based audio engine to be processed there. And then, oh, but now at this point, I want to take over again, and then the Python code will receive these events again.

That's cool. This kind of stuff. I personally had a lot of fun when I was working on, you know, two halves of a system like that. So I expect that it is a fun project to work on. Yeah, that's pretty cool. It can be sometimes also a bit mind-bending, like this between two languages kind of thing.

Right. But yeah, it's a lot of fun also. We're using Boost Python also to expose our C++. Yeah, exactly. And yeah, it's kind of also a double, how do you say, two-sided sort. Right, right. It cuts you no matter what you do with it.

Yeah, yeah, exactly. I mean, it's really powerful and you can do a lot of things very concisely. But if you get something wrong, then you get these really bad template metaprogramming error messages where you don't really understand what's going on. And it's kind of impossible to debug

because it's all happening at compile time. I might get some hate mail from Boost maintainers for this, but it feels to me like Boost Python has been largely unmaintained since it was finished. I was just going to ask if you looked into any of the other libraries. I mean, we've had John Heed on the show a couple of times who maintains Sol, which I think is... That's Lua.

That's Lua. My mistake. But yeah. Yeah, that's possible. We're actually interested in migrating to something called Pybind 11, which is based on C++ 11, so it has to do less weird macro hacks and stuff like that, which is nice. But yeah, that's not something you migrate overnight with a big code. No, no, no. It's one of those things where once the code is working, does it really make sense to migrate to a whole new binding package?

You have to have a good reason to, I feel like. Someone prototype and demonstrate that's actually 10 times faster, well, then maybe you'll do it. Yeah, I mean, that would be a good reason. Yeah. All right. Well, Nikolai, we've got a couple of news articles to discuss, and then we'll start talking more about the gaming work you've been doing, okay? Sounds great.

All right. So this first one is a blog post from last week. Jason, I guess we recorded the day before April Fool's, so we didn't have any of these lovely uh april fools jokes to talk about last week um but this one was about uh how c++ 23 uh the arrow and uh colon colon would be replaced by the dot operator which is basically uh what c sharp has and of course this was an april fools joke post i will say it is i mean it, it's kind of what ChaiScript did too, because you could do a dot and it would, if you had a

reference to an object, it was like, fine. If you had a pointer to an object, it was like, whatever. It would just resolve that at runtime, which was what to do, which makes sense in some ways for scripting language interaction, but yeah. I was only like one sentence in before I realized that it was an equal fools

one proposal af001 yeah yeah that was definitely the giveaway at that point yeah i actually learned something from this article which is that apparently if you have a like old school enum not an enum class you can refer to the constants in the enum via the dot if you have like an instance of a class and inside of the class you have an enum declared and it has like a b and whatnot then you could do instance dot a and it would give you the value of that constant with which is um interesting that is interesting yeah i would think I wonder if some compilers would give a warning saying that you're accessing a static value with a dot. I think there's some warning flags for that.

Something in Clang, maybe, yeah. I've never tried it myself. Okay, next thing we have is a post from the Old New Thing blog, Raymond Chen, and this is C++ Coroutines, the mental model for coroutine promises. And I think we talked with Raymond a little bit about coroutines when we had him on the show. But he's just started this set of blog posts all about coroutines where he's going to be going into

some details about how they work, how you can describe them. And I think this is kind of the start of that. I really appreciate that. This is like a bite size article. It's not like here are the 53 things that you must know to use coroutines. It's like, okay, let's just start to get a mental model for coroutines. Yeah. And then he can continue building upon that. And like I said, he posts every day, so I'm sure they'll be going into more depth but uh yeah it's good that he starts off small because coroutines are a complicated

subject so the one we're linking to is from the 29th and yes there are definitely already 30th and 31st and first that continuing this posting series right nikolai you have any thoughts on this one i haven't spent much time with coroutines myself yet um but i'm interested in it also um with the game project there's some things where i wonder if you could simplify some stateful code using coroutines so yeah i haven't really looked into it in detail but um yeah I want to play around with that at some point. I don't know, are they already available in all compilers at this point? I don't think GCC has them yet. Sounds right.

Or is it the other way around? Well, definitely Visual Studio, and Clang was where they were first prototyped. Right. Okay, and then we also have this GitHub repository repository jason you want to tell us about this i just uh my cousin pointed this out to me um my cousin john turner who we've had on talking about rust in the past uh that there is someone re-implementing the rust c compiler in c++ and i'm like okay we have to just talk about this but i will admit that i didn't actually look at the source code until about an hour ago and i'm gonna do this this code is written in c++

air quotes for anyone just listening yes uh there's a lot of macro usage there's a lot of c++ 98 isms like explicit throw clause instead of saying no except throw with empty parens and as far as i can tell it's code that's only honestly it's five years old six years old so i'm slightly confused there but um yeah and the the author is is making it pretty clear that this is definitely a work in progress and that you'll probably see lots of really, really ugly errors if you try to compile using his Rust compiler, right? Yeah. But if you try to compile something that's not valid code.

Right. It assumes all code is valid. Yeah. Okay. I do think this is, like like honestly healthy for the the rust ecosystem though because they only have one reference compiler right now and when you have someone else try to implement the spec that's when you like really find the bugs yeah that's true yeah totally i was wondering actually what form the the rust spec has or if there is an official spec,

even if it's just like, oh, what the compiler does. I believe that there is official documentation, but it's obviously, I mean, it's not an international ISO standard like C++, so it doesn't have the legalese written out. I know I've asked about that in the past. Now I can't remember the specific details. Okay, and then the last thing I wanted to mention I know I've asked about that in the past. Now I can't remember the specific details. Okay.

And then the last thing I wanted to mention is that we have this post from the Microsoft C++ blog, and this is from Cy Brand, who we've had on the show many times, announcing that they're going to do another pure virtual C++ conference this year. And this is going to be on May 3rd, and it'll be on YouTube and Microsoft Learn TV.

I know I watched some of the content last year, and it was really good. I mean, it was great to have that virtual conference kind of early in the pandemic when we were seeing all the real world things shutting down. So it's nice to see them continuing that again this year. What the heck is Microsoft Learn TV? I was wondering the same thing because i've heard of channel nine yes but i don't know what the relation is between this learn tv and channel nine it seems to be it's a different website i want to see if channel nine affords you to learn tv it does not they are oh wait no it does have a banner at the cross the

top saying go to learn tv so channel 9 is telling you to go to learn tv oh yeah yeah but it does also still exist in its own is it like the pure virtual c++ 2021 conference also has a channel 9 link so the the link on channel 9 going to learn tv it says 24 7 developer stream so is this like twitch for developers is that the idea i don't know i was thinking about making like a c++ tv channel you could just tune in and just watch whatever happened to be on at the moment i mean you could do that on twitch or youtube streaming right just queue up the next thing repeatedly. I don't know if people would actually watch that. Okay. All right.

Well, Nikolai, let's start talking about the Rigel engine that I mentioned in your bio. Can you tell us what that is exactly? Sure. So Rigel engine is a re-implementation of the game engine for the game Duke Nukem 2. And Duke Nukem 2 is the predecessor to Duke Nukem 3D, which most people are probably familiar with. But before that game, there were two other games that were 2D games, 2D side-scrolling games,

where you kind of just run around and jump and shoot things. And Duke Nukem 2 in particular is 27 years old by this point. Oh wow. Yeah. And it was released for MS-DOS back then. So you can play it nowadays in an emulator like DOSBox. But now you can also play it using my project, regular engine.

And so that's basically written from scratch in C++17, new executable that you can kind of drop in as a replacement for the original DOS binary. And then it reads the game data, like the files, with all the graphics and sound effects and the levels and all that. And yeah, then lets you play the game

natively on a modern machine. I'm a little disappointed, just for the record, that I don't see big box PC games behind you, since you're talking about Duke Nukem 2 right now. I expect to see a hard copy back there. I actually have one. It's just out of view a bit. I'll take your word for it. yeah okay awesome i feel like you had to own one just because right pretty much yeah i mean i i actually only got this once i started this project so i never had the full version um back as a kid i played all these shiver versions like back then you had this shiver distribution model where you could freely download parts of the game and then you would have to buy um the rest and yeah but um once i started the project i was like man i really want to have the the original game and then

tracked it down on ebay and yeah and probably paid too much for it way too much way too much i don't want to say how much. But yeah, I was surprised the floppy disks actually still worked. Oh, wow. So you did go ahead and install the original version? I did. I also have some period-appropriate hardware. Okay.

Like some old 386 and 486 PCs. And so, yeah, I did also try it out on these okay so I'm curious if you have like a USB floppy drive that you read it off onto your main PC okay I've heard mixed things about those USB floppy drives sometimes they're like oh yeah I did exactly what I needed to and sometimes it's like it ate my floppy

oh wow okay yeah I didn't have any problems with this one um seems to work fine apparently i was surprised to learn this um don't know if uh you're familiar with this twitter account um from this person called foon f-u-u-n-e uh yeah that name's familiar yeah they do a lot of reverse engineering of old games. And they also run this thing called DeathGenerator.com. DeathGenerator.com. Yeah, it's from the Sierra games where you have, like, all these ridiculous death scenes. Oh, nice.

And so it lets you input your own text into these kind of game scenes and make your own kind of funny images. Nice. Yeah, and so they always post about funny and weird old computer stuff. And now I kind of forgot where I was going with this. Ah, yeah. So they once had a thread where they talked about that even to this day, Windows 10 still has some drivers for floppy disk access, apparently. And that's how these USB floppy drives work. I was just watching one of these retro hardware episodes yesterday uh where someone was pointing out that if you have a ide zip drive you can just plug it straight in if you have an

ide adapter on your motherboard and windows 10 still has no problem just accessing a zip drive. Interesting. Yeah. Wow. Unlike some manufacturers that drop support for things a week. Right. Some manufacturers that will remain unnamed. So what exactly got you started? What inspired you to go and want to you know rebuild duke nukem right um so it's kind of two things that led to me starting the project um so i discovered some of these other game re-implementation projects i think the first one i discovered was something called omni speak which is a re-implementation of the Commander Keen game engine. Okay. I remember when I was a kid. Yeah, another classic game.

And so, yeah, I found this post on Reddit about it. And at the time, this was the very first project of that nature that I've heard of. And I almost couldn't believe it, that someone would be able to, from the disassembly of a binary, recreate an entire game. But then, and that it would actually work super well. But then I discovered that there's a whole kind of scene

of people who do these little reverse engineering projects. And there's a website called osgameclones.com that lists a lot of these kind of fan-made game engine recreations and it's really tons of them i was surprising um yeah so i had discovered this and then that wasn't the concrete uh thing that led me to making my own project but um this was in 2016 when i started the project um and i was on vacation and felt like doing some fun little coding stuff. There's something wrong with us that we're like, we're on vacation.

Let's do a fun programming project. Yeah, let's do the stuff I do at work now in my free time. I know, right? But yeah, I wanted to do a little bit of programming and I found this wiki, the Shikadi modding wiki, where people collect all kinds of reverse engineering information for file formats and stuff like that for various old games.

And I've also in the past played around a bit with this file format reverse engineering um before and so i felt like oh yeah i could do some reverse engineering and implement some like graphics extraction stuff for some of these old games and ended up implementing a sort of level viewer that would be able to show the levels of a bunch of these old RPG games so like Commander Keen, Duke Nukem and some other ones and originally that was kind of where I was going and I thought oh maybe it would be cool to make a level editor even out of this that would support different games and you would be able to edit the levels for all these games

but then I showed it to a friend and he kind of misunderstood what I was going for and thought that I was trying to recreate the game which at that point wasn't really my plan but then that combined with having a while before discovered all these other

reverse engineering projects made me think, Oh, well, maybe this is actually something that would be fun to do. And yeah, so then I kind of took this work that I had already done for this level viewer and took the Duke Nukem specific parts of it. And this was, so this level viewer was done with using Qt and was more like a desktop application. And then I took this code and took SDL and have made a full screen renderer that was a bit more like game-like and try to get the level display level rendering

to work more like it worked in the actual game. And that was kind of the start of the project. So you got accidentally nerd sniped, basically. Yes, pretty much. Like, I wasn't planning to, oh, crap, now I have to implement the game. Yep. And now five years later, I'm still doing it. That's awesome.

So you said you had this level viewer. It wasn't just Duke Nukem, though. You said a bunch of other games, right? Any plan to go and now implement the game engines for those other games, too? I mean, you were planning to do that in the first place, right? Kind of, yeah. Although maybe the scope is a little different.

I mean, for lots of these games other people have also done it already right so like there's two projects for commander keen that's one project for cosmos cosmic adventure it's another one of these games there's one for duke nukem one and yeah so sorry go ahead was part of the appeal of it that it was a game that no one else had already made an implementation of I think so that and also it's one of the my favorite games from back in the day basically right and yeah I've played it a lot and enjoyed it a lot is it still fun to play now that you've written your own implementation of it? I'm not sure. So it's still fun to, I mean, I still enjoy a lot working on this project and firing it up and like testing some things.

But I'm not sure I could still really sit down and play, just play the game. I think I've played it too much by now. Testing all the stuff and yeah i'm not sure probably not so you said you've been working on this for for five years how long did it take before you had you know something that you could actually go and play that was written you know more modern c++ right so it didn't take that long to have like a very basic thing okay where you could walk around in levels but then to the point where it fully supports the game i only i got i think so where the entire game is really fully supported that was summer last year oh wow also took like four years yeah okay and then a year

before that i kind of got the shareware version only working so that's like the first eight levels oh interesting and i pretty quickly got to the point where you could play the first level but then um all the other levels had some movement mechanics that were more difficult to implement or had more enemies that I needed to implement. And so that took a while then. And then what made the commercial version different from the shareware version that made it harder to implement

or what was still left? Basically just all the enemies. So in this game, it's different for different games, right? In some cases you have a very low level engine and then almost the entire game is kind of written in a more scripting language kind of way. But in this case, most of the game logic is also implemented as native code in the binary.

And so the behavior for all the different enemies and what happens when you do certain things is all kind of part of the native code in the binary. And so the commercial version had additional enemies, additional game mechanics. There's like a spaceship that you can jump into and fly around in. So it has some new movement mechanics and yeah mainly that a couple more game mechanics and um and enemies and also then fixing

bugs because it turns out that some of the game mechanics i got working to a point where you could play the first couple of levels without issue but then the other levels from the commercial edition would have slightly different situations or slightly different arrangement of things where i would then discover some bugs that made it sometimes impossible to progress in a level or things like that i'm i had i never really played duke nukem but i'm imagining like in super mar Mario Brothers or something where a particularly tricky level, you have to run and time a perfect jump

and it's like pixel perfect accuracy to be able to land on the other ledge. And I'm imagining that's the kind of thing that you're talking about. Yeah, kind of. Also about collision detection, like there were things where, so there's a mechanic where you can kind of climb on pipes,

where he's hanging from this with his hands, and then you can climb left and right, or move left and right while hanging on this pipe. And there were some little details to how exactly that worked. At what point do you kind of fall off the pipe because you've gone too far to the left or right, this type of thing. And where this would then also have some interplay with the collision detection where I could then suddenly get stuck in walls

because they had slightly different level geometry than some of the other levels. And then that triggered some edge cases that I hadn't really considered. Things like that. Sponsor of this episode is the PVS Studio team. The team develops the PVS Studio static code analyzer. The tool detects errors in C, C++, C sharp, and Java code. When you use the analyzer regularly, you can spot and fix many errors right after you write new code. The analyzer does the tedious work of sifting through the boring parts of code. Never gets tired of looking for typos. The analyzer makes code reviews more productive by freeing up your team's resources.

Now you have time to focus on what's important, algorithms and high-level errors. Check out the team's recent article, Date Processing Attracts Bugs or 77 Defects in Qt 6 to see what the analyzer can do. The link is in the podcast description. We've also added a link there to a funny blog post, COVID-19 research and uninitialized variable, though you'll have to decide by yourself whether it's funny or sad. Remember that you can extend

the PVS Studio trial period from one week to one month. Just use the CppCast hashtag when you're requesting your license. So you want to tell us a little bit more about the actual process it takes to go about reverse engineering game? Like you said, there's this online community of people who disassemble from the binaries. How do you go from that disassembly to writing new code?

Right. So there's a few different approaches that I've seen. What some people do is that they really try to recreate the original source code to the point where they end up with source code and you have an identical binary. That's not the approach I took. So the way I did it was more to use the assembly code to try and understand how things are working. And then depending on the complexity, in some cases it was easy enough

to just look at the assembly code and be like, okay, yeah, so it has some kind of counter and it's just incrementing it every frame. And when the counter reaches a certain value, then something happens. Sometimes it's a lot more complicated. And then in those more complicated cases, I would manually transcribe

the assembly into some sort of pseudocode that was C-like, but not quite. And then based on that, it was easier to actually understand what's going on and then implement my version based on that. Wow. And then what I also did was to help with understanding some really tricky things i took dos box and modified it so that it would write out a memory dump every frame and because it's a dos game you can actually do that because the entire address space is just 640 kilobytes right yeah so on a modern machine it's not that much data and yeah it sounds like just unbelievable putting it in the historical perspective for sure like i just dumped the whole

memory of the machine every frame why not yeah exactly exactly and then but then so i had these memory dumps and then because i knew the layout of um where all these different variables because it's mostly global variables, the way the original game works. Global variables and a bunch of static arrays that are used to hold objects, where they kind of, they pre-allocate a big array and then kind of have slots for objects

and they are marked as deleted if they are not used. And then this memory can be reused when you add a new object. So it's kind of like a pool in a way, maybe, a memory pool. But a very, yeah, very basic way of doing it. Yeah, but because I knew how all this memory layout was, I could then analyze these memory dumps and figure out, okay, in this frame, this variable has value,

I don't know, five, and then the next frame, it's suddenly 10, and okay, let me try to figure out why it changed from that to that. And this kind of helped in some cases where I looked at the assembly and was like, Okay, I think this is doing a certain thing, but I'm really not sure this seems really weird that it would do that. And then it could kind of verify these things. And in some cases, I also discovered really weird things in the code. I don't think these old compilers did very much in the way of eliminating dead code because there was often branches that were unreachable,

but they were just still in the machine code for some reason. I believe that. Yeah. Yeah, even in the early 2000s, I know of compilers that had bugs around dead code elimination. Although it was the other way. It eliminated code that was actually still in use.

You know, it's fine. Wow. So it sounds like you're kind of set up for a whole now second career in reverse engineering binaries and looking for vulnerabilities. Are you doing bug bounty kind of research bounty i guess i could yeah i mean um i think that's still a whole nother level if you want to do that with um like modern applications yeah yeah i mean already i have to say the difference between modern x64 assembly code that optimizing compilers produce and this code from back then, which was 16-bit x86. And so it had some quirks that make it more difficult to understand, but it's also in many ways much more straightforward. For example, the calling convention is really simple.

It's always just pushing all the arguments on the stack and then doing a call instruction. And so you can almost read the function calls just by looking at the assembly. Right. Less register tracing. Yeah. Much less of this complicated stuff. And yeah, in general, much less optimizations. And it's often very straightforward to see,

okay, this assembly is and then this assembly and then what was kind of the c code that produced this assembly which if you look at modern compiler output is yeah a bit different different story are you familiar with how some of the modern like nintendo and Commodore emulators and stuff, you can actually do, like, live view of the memory, the register state, CPU state, whatever. And it just makes me wonder, like, someone should do a fork of DOSBox that has those features, just lets you pause at any moment and see all the things.

That would have been really helpful to you, I think. It would, yeah, for sure, yeah. Yeah, that would be nice. You should do that. would be nice you should do that yeah maybe i should do that actually i probably wouldn't technically be that difficult because the whole state's going to be visible anyhow you just have to add a gui around it and decide how often to update it without slowing down the rest of the machine yeah pretty much yeah that would be an interesting project.

So you mentioned how some people in this type of community like to try to faithfully recreate the code, but you're going for more of a modern style. What type of C++ code is your implementation in? Right. Yeah, I think the way I approach things are kind of making it a bit more difficult for myself in some way but also it's more satisfying so i'm really i do want to um make the game behave really exactly like the original sure um in terms of like the user visible behavior. But the code, I want to be much more modern. And so concretely for me,

that means I want to avoid global state and I want to try to in general manage state, like use const variables a lot and go for a more functional approach where it's feasible to do so. I'm using std variant a lot and this kind of pattern matching or pseudo pattern matching you can do with std variant where you have a variant of different types and then they that can represent a state machine for example and you then have this overload utility

where you can specify lambdas to handle the different types in the variant. And then a lot of the enemies are essentially state machines that are like, oh, walk to the left for a couple frames, then shoot at the player, then turn around, this kind of thing. And so in the original game, they do that mostly based on...

Sometimes it seems like it's switch statements, but most of the time it's really weird spaghetti code with tons of branching all over the place and maybe even some go-tos, I'm not sure. And so I'm trying to make that more structured and a bit more to a point where it's a bit clearer what the flow of execution is and where it's less convoluted now with your already having experience with boost python did you consider making some of these uh

game logic things actually scripted so that it was easier to tweak and change and stuff yeah yeah i definitely thought about that um and in the end i didn't do it um because yeah it felt um felt right to just do everything in c++ okay but it would be an interesting approach for sure and i think it would also make some of the game logic code more concise and easier to understand, I could imagine. You started the conversation by saying, and Rob just brought this up too, the game community sites that are already out there where people have done reverse engineering of the game files and that kind of thing. So two questions related to that. Was there already reverse engineered versions of the duke nukem 2 game files

and is there already a modding community out there where people are making their own duke nukem levels and stuff um so there's this wiki which has a lot of information about the file formats and there are some tools that can read these file formats already out there um in terms of mods it doesn't seem like there's very much of a active community definitely not and i found maybe one or two things but nothing really super big so like even for commander keen there are some projects where people have done these total conversions where they completely change how everything looks. But I haven't found anything like that for Duke Nukem 2 so far. I'm kind of curious, having read the Masters of Doom and John Carmack's approach to programming

and some of the things that we take for granted today, like scripting logic in the games and making the WAD files for Doom and whatever, and knowing that commander keen was one of carmax earlier games i'm kind of curious if there's some reason why keen has been more modded if he made it more of a moddable game in the first place i think so if maybe unintentionally so but one difference i know is that and i might have some of the details wrong but i think the way it works in commander keen is that um enemy behavior and animations

are kind of defined as data so it's still compiled into the executable right but it's not code it's um data that's then interpreted to be like okay, and you can kind of build up these kinds of animation sequences and state machines that way. Yeah, that's how it is in Doom also, yeah. Yeah. Whereas in Duke Nukem 2, it's really just, it's all native code, and kind of bespoke update functions for everything. And so the only thing you could really mod in Duke Nukem 2 is like the graphics, but then you still would need to stick exactly to the number of animation frames, for example. So you couldn't make like an enemy that looks different

and has some different animation. But you can make a new level using enemies that already existed. Yeah, you can definitely do that. And yeah, there are some level editors also out there for it. Okay. So the game is more or less done now? Are you still working on it? Are there bugs you're fixing or anything like that?

There's a pretty funny bug that my girlfriend just discovered the other day, actually. I mean, maybe yeah to first answer the question so the core functionality is pretty much done so you can play the entire game um but there's still a few minor missing features um some of them are just visual effects for example there's this kind of invincibility power-up that you can collect in the game that turns your character sort of transparent.

And in the original game, they implemented that with some funky... So the original game is in a 16-color mode where you have a palette of colors, and then they kind of rewrite the frame buffer to change the color indices and this results in a sort of translucent look and so i could recreate that in a shader so then i'm using opengl for rendering i could recreate that in a shader but i haven't um done that yet and so in my version it's just a alpha

set so that it's kind of transparent to replicate this effect which doesn't quite give the same effect but also doesn't really make the game unplayable or anything so it wasn't that high on the priority list um yeah the funny bug is so there's an enemy that that's a snake a giant snake and it can eat you when you touch it and when it eats you then you start taking damage and you're kind of inside of the snake and it's like crawling around with a you inside of it but you can still fire your weapon and that then kills the snake from inside and you're free again And so the way this is implemented is that when you touch the snake, it kind of puts the player into a special state where it's not rendered and you cannot control the player. And now it turns out that there's a very rare edge case where if you kill

the snake on the exact same frame that it eats you, then the snake is not yet in the state where it would run the logic to set the player free again when it's killed. So what happens is the snake disappears, but the player is also still gone. Still in the snake, so to say. And you can't control it. And then essentially the game is not crashed or anything,

but you can't play it anymore. It's kind of of stuck and this is a bug that does not exist in the original exactly and it's kind of yeah because i didn't uh take this edge case into account are there bugs that you did oh sorry uh yeah go ahead are there bugs that you have had to intentionally recreate like you accidentally fixed a bug and then you had to add it back in? Yeah, there are actually some. And so, yeah, this is another one of the missing features is that the original game has this kind of pre-recorded demo, this kind of attract mode thing,

where if you wait on the menu without any input, then it starts playing by itself and you can watch how the game looks and the way they implemented this is that they just record the key press events and then they have the actual game running but feed it with these pre-recorded input events instead of the actual input and so for that to really work um my version really needs to be 100 perfect uh matching the original implementation because otherwise it gets out of sync um and so this is not quite there yet okay because of some differences in how and mainly so i took a lot of time and put a lot of effort into making all of the individual enemies and game mechanics really exactly like in the original and i also um did these video comparison with comparisons with dosbox where i would record my game and then

dosbox and then put them side by side and step through them frame by frame. So on that level, it's really quite accurate. But there are some differences in the order in which things are updated. And then because there's also random numbers involved, and the random number generator repeats very quickly, and is in general not very random. It's extremely pseudo-random. It's like just an array of 256 numbers that it goes through linear like a hard-coded array of 256 numbers oh wow

nice that reminds me of that dilbert 999 that's our generator yeah there's also an xkcd where there's a function that's like return 42 and a comment no not 42 return five or something and determined via a fair dice roll right right right reminded me of that as well um yeah and because of this um the how the game behaves depends really on um does every object in the game get exactly the sequence of random numbers that it gets in the original game? And since, so I think I mentioned before, the original game is using kind of these pre-allocated arrays of objects to manage everything. And then if you destroy an enemy that's at an early position in the array,

it kind of gets marked as deleted and then if something else is created like maybe an enemy shoots a rocket at you then this rocket will be take over the spot in the array at the beginning of the array but if there's no free spot it will appear at the end of the array and so kind of the order in which this rocket is now going to be updated and receive random numbers depends on kind of the situation of what happened before and how this array is laid out in the original game. And so I didn't recreate that behavior in my version. And that's one of the things that makes the demo not really be in sync. Right. the demo not really be in sync. Right, right. So you mentioned that you did the rendering in OpenGL. What platforms can you play Rigel on?

Right, so right now it supports Windows, Mac, and Linux. Okay. And then anything that has a graphics card that supports either OpenGL 3.0 or OpenGL ES 2.0. So you can also run it on a Raspberry Pi. And there's a WebAssembly version also. It runs in the browser also. It seems obligatory at this point.

Yeah. Yeah, that was actually contributed by someone. Oh, very cool. Have there been a lot of contributors on the project? Not that many. So there's a lot of contributors on the project? Not that many. So there's a couple of people that kind of showed up and did a thing and then disappeared again. There's one person

who's pretty regularly done things. Yep, go ahead. What new skills did you develop simply because of working on this? I think definitely when it comes to the reading assembly and this kind of working my way through a binary. So I'm using IDAR Pro, which is this assembly tool,

which is nice because it shows this graph view where you can see the control flow a bit um better than if you were just looking at a flat list of instructions um yeah so definitely that and also i learned a lot about the way you had to program on a dos machine and like all these hardware interfaces with the vga card and the sound card and this kind of stuff which i mean it's not so relevant for most of the things but when it comes to some of these um special effects like this invisibility thing that i mentioned there's also another effect that i have already re-implemented as a shader which is some kind of underwater effect where it

makes everything bluish in a certain area. And so for these effects to figure out how exactly they worked, I kind of went down into the guts of the rendering code of the original binary. And then I had to learn about, okay, how does a VGA card work? What does it mean if you send these values to these registers? And yeah. Wow. So yeah, it's been super fun i'm not

sure how applicable these particular skills are in the modern day had you ever created a game before this um a few small ones okay so never i'm i've never worked professionally on game development but um yeah i mean i think when i started programming it was mainly motivated by wanting to make games i think when i got what really got me serious about getting into programming was um i think it was my dad got me a present for my birthday once which was a book about making games with direct x okay and c. And yeah, this really hooked me. And I made some kind of Space Invaders clone at that time based on this book.

That was kind of the first game I did. And then, yeah, a bunch of smaller projects here and there. Yeah, and the course of this interview, between the time that you said that you're using SDL and the time that you said that you're using actuallyL and the time that you said that you're using actually OpenGL for the game, for the graphics rendering, I was really, really hoping that you had taken the project full

circle and managed to recompile it for DOS with DJGVP today, which is still being updated. You can get GCC 10.2 for DOS. And run it with a Voodoo card, yeah. Yeah. I think you should go for it you should uh you should port it back to dos now jason's just trying to sign you up for so much more work i know i've got like four projects to work on at the end of this interview okay well nikolai it's been great

to have you on the show is there anything you want to plug or anything like that before we let you go where can people find you online uh right so i mean that's my github account um which is lethal guitar um lethal underscore guitar no lethal dash guitar it looks like yeah um on twitter it's lethal underscore guitar i think yeah yeah um also have a blog where i very infrequently um post things about the game which is at um lethal guitar without any dash or underscore dot webpress.com yeah it's weird um all these different um platforms have their own rules about which you can have a username. Right. You can't have...

Are you doing YouTube videos also? I do have a YouTube channel which has some videos about the games, but these are mostly like there's a video on the GitHub page that I link and on the blog. So yeah, I don't post super regularly, but that's also, that's at, what's my name on YouTube? Lethal Guitar 128. Without underscores. We'll make sure we get all that in the show notes. Niklas, great having you on the show.

Yeah, it was a pleasure. Thank you for having me. It was a lot of fun. Thank you. Thanks. Thanks so much for listening in as we chat about C++. We'd love to hear what you think of the podcast. Please let

us know if we're discussing the stuff you're interested in or if you have a suggestion for a topic. We'd love to hear about that too. You can email all your thoughts to feedback at cppcast.com. We'd also appreciate if you can like CppCast on Facebook and follow CppCast on Twitter. You can also follow me at Rob W on Facebook and follow CppCast on Twitter.

You can also follow me at Rob W. Irving and Jason at Lefticus on Twitter. We'd also like to thank all our patrons who help support the show through Patreon. If you'd like to support us on Patreon, you can do so at patreon.com slash cppcast. And of course, you can find all that info and the show notes on the podcast website at cppcast.com. Theme music for this episode was provided by podcastthemes.com.