CppCast - Visual Effects

Episode Date: July 30, 2020

Rob and Jason are joined by Josh Filstrup from Netflix. They first discuss an update to wxWidgets and a bug fix update in GCC. Then they talk to Josh about his background in Programming Languages and how he transitioned to work in the Visual Effects industry, including his current job at the studio engineering group at Netflix. News wxWidgets 3.14 Release GCC 10.2 Release C++ On Sea videos coming online Links USD VFX Platform Resurrecting the SuperH architecture DCDigital - REAL Dreamcast 1080p HDMI Output CppCon 2014: Chandler Carruth "Efficiency with Algorithms, Performance with Data Structures" Sponsors PVS-Studio. Write #cppcast in the message field on the download page and get one month license PVS-Studio is now in Compiler Explorer! Free PVS-Studio for Students and Teachers

Episode 258 of CppCast with guest Josh Philstrup recorded July 30th, 2020. Sponsor of this episode of CppCast is the PVS Studio team. The team promotes regular usage of static code analysis and the PVS Studio static analysis tool. In this episode, we discuss bug fixes in GCC. Then we talk to Josh Philstrom from Netflix. Josh talks to us about his background in programming languages and work in the visual effects industry. Welcome to episode 258 of CppCast, the first podcast for C++ developers by C++ developers. I'm your host, Rob Irving. Join my co-host, Jason Turner. Jason, how are you doing today?

I'm okay, Rob. How are you doing? Doing okay. Any news or anything you have to share? Not particularly. I'm planning to do some more streaming on my YouTube channel today and tomorrow, but this won't air until tomorrow and it's too late for people to tell anyhow. But you have to be subscribed to me on YouTube if you want to know about these last minute streams. So I guess I can just make that as a plug, I guess. That's a good plug. What exactly are you doing on the stream today uh i'm going to be recording um some hacking on the game engine stuff that i started about seven months ago at the beginning of the year because

it hasn't been moving as quickly as i would like it to on the channel so i just figured i'm just gonna stream i don't know like maybe 10 to 15 hours of stuff over the next couple of days while I just hack on it and see if anyone cares. Cool. Very cool. Okay. Well, at the top of the episode, I'd like to read a piece of feedback. Speaking of YouTube, we got this YouTube comment from an episode we did several months ago. Someone's catching up on it, I guess. This was the Commodore 64 and Tilt 5 ar episode with jerry ellsworth and riley commented you should team up and make new baby commodore 64s and new games for them which i think is aimed

at you jason uh perhaps that's i mean that episode is so much fun and what he's referring to just in case our listeners don't remember is on air jerry said that she had something like a hundred thousand dies left yeah from the camera sitting in her closet yeah from this project the mammoth games commodore 64 joystick that i still have sitting on my desk right now right um which is a system on a chip commodore 64. It would be awesome for those things to see the light of day. Yeah. Yeah, definitely.

Have you been tracking that Kickstarter as that started shipping out yet, the Tilt 5 stuff? I remember a few months ago seeing notifications that it was starting to ship, but I hadn't checked again after that since I wasn't one of the, I didn't buy one.

No, I didn't either. It was a little expensive, but I'm looking forward to seeing how well it does. Yeah. I know several of our listeners did because we got tweets and stuff, people talking about having modded. Right. Okay. Well, we'd love to hear your thoughts about the show. You can always reach out to us on Facebook, Twitter, or email us at feedback at cpgas.com.

And don't forget to leave us a review on iTunes or subscribe on YouTube. Joining us today is Josh Philstrup. at cpgas.com and don't forget to leave us a review on iTunes or subscribe on YouTube. Joining us today is Josh Philstrup. Josh is a senior engineer working in the visual effects industry with a focus on building creative tech that scales. Currently, he works in the studio engineering group at Netflix. Previously, he has worked at Disney Animation and Pixar. His computing interests are in programming languages and high-performance computing and lately exploring the history and specification of old game console hardware. Josh, welcome to the show. Hi, thanks so much for having me. I feel like you put that last sentence in there just so I would ask you about it.

You know, it's funny. I made the connection just as you were talking about Commodore now. I hadn't thought about it, but it's really that during the inside times as i'm calling them i have found many new hobbies and one of them is exploring obscure hardware okay what's the most obscure hardware that you've been researching so i've been spending a lot of time thinking about the sega dreamcast lately like probably far too much time thinking about the Sega Dreamcast. The one rabbit hole I went down was, so they had this processor called the Super H processor. And

it was pretty big back in the day and a lot of embedded devices apparently. And it turns out that there's this group in the open source community called like the J-Core group that did a clean room implementation of like that instruction set on a new open source chip.

Because apparently the patents for the second SuperH processor, the SH2, just expired. So they're able to do these clean room implementations. So it's this crazy, crazy rabbit hole you can go down. And there's just a lot of interesting obscure hardware. The Dreamcast in particular is interesting to me just because it did so much that was ahead of its time. One example is it had a broadband adapter. And mind you, this is like four years before the PlayStation 2 really did it with Final Fantasy XI. And it just, yeah, the Dreamcast was really, really weird.

It did a bunch of advanced stuff that was one of the first of the 32-bit era right is that right the dreamcast so the dreamcast came out uh before playstation 2 um so it was 99 1999 in uh america i think it was about a year before or so in japan okay um But it had power that rivaled the PlayStation 2 GameCube era. Awesome. So are you then now implementing emulators or did you buy an FPGA kit

so you can have your own? What are you doing? So I started with restoration. So I had a really old couple of Dreamcasts that my mom mailed to me because I was like, I don't have anything to do. Can you mail me those broken Dreamcasts?

So I've been repairing those and one of them had a chip that essentially unlocked the BIOS to play games from any region. And so I was like, oh, this is really cool. If I can make it work again, this would be really fun to play with. So first it was learning about

proper plastic restoration and all these like hardware tricks. And then soldering new fans in and all these kind of parts that typically die. One part that's like super common to die in there, they had this thing called a GD-ROM inside the Dreamcast. And the GD-ROM was like a CD-ROM, but it had a gigabyte disk. So it's standard for gigabyte disk. Okay. But it was like this, you know, proprietary format.

And so like when these things die, there's kind of no saving them. Like it's like, cause only Dreamcast had GD ROMs, like nothing else had it. Whereas like a CD ROM drive, you can kind of find them all over the place. Maybe you can kind of salvage it.

So essentially you end up Frankenstein parts out of GD ROMs. Other things that fail on them are super common, like power supplies and things like that. So you just end up replacing all the capacitors and you're good to go. So it was a lot of hardware stuff like that. Now on the software side, I've moved on to doing emulator implementation, but I'm starting small and just emulating the CPU itself. So I'm just working on a SuperH emulator. And then it's like, if that goes well, maybe a dreamcast emulator.

That's awesome. So wait, wait, but does the, does the dreamcast work yet? Oh yeah. Both work. So I have two and they both work. They both work. One is completely OEM and the other is much more standard new parts. Like people have developed new custom power supplies for them, new emulator chips that replace the GD-ROM. And people have even gone so far, if you look up a thing called the DC Digital

by these two really brilliant hackers, it's an HDMI FPGA that solders directly to the motherboard and gives you true 1080p output. Like, it's insane the level of, like, things people have developed for these old consoles. So one of them's original and the other one has all those things installed. Yeah, yeah. One of them's original and one's the one I actually want to hook up to my TV. Did you have to recap all of them?

So you don't, like, the motherboard was pretty good on the one that's hooked up to the TV. You don't have to, but, like, people sell full kits, so I'm planning to at some point soon, but I haven't gotten around to it yet. I'm just thinking about your setup personally, because I invested in a decent soldering iron not that long ago, because a decent soldering iron is not that expensive, but a decent desoldering gun with a vacuum pump in it, I just can't convince myself to spend a couple hundred

dollars or whatever on one of those because i would only use it like twice i know i wouldn't you know yeah i actually just got a nicer soldering iron myself as i started getting into this because i had like a 12 one that was like no temperature control right it didn't even have a stand to put it so i was like precariously setting it on the table and hoping to not burn myself um and so like i at least got like a nice one with a stand now but i i also can't go that far because it's like i feel like i'll use it still like maybe 10 times a year yeah yeah maybe yeah okay it's something we never talk about on this show really yeah we've gotten into hardware for a while. Okay.

Well, Josh, we've got a couple news articles to discuss. Feel free to comment on any of these, and then we'll start talking more about the work you're doing on Netflix, okay? Cool. All right. So this first one is an update to WX Widgets 3.1.4, which is the first update in, I think, nine months months and we did have an episode with WXWidgets but that was the episode

where we had some pretty poor audio quality so I feel like we should maybe do a follow up again and talk to someone from WXWidgets soon maybe still on my to-do list, yeah but a couple of notable things in this one are they're able to compile with C 20 now, which is great.

Some CMake build improvements. Anything else you wanted to call out here, Jason? There's this one line item right here. WXOS export now supports the latest Mac OS 11 on ARM hardware. Now, as far as I know, no one has access to a Mac OS 11 on ARM hardware. So I'm really curious exactly how they tested that and what that means. Those lightweight MacBooks, like MacBook Air, that's not on ARM? No, that's not on ARM.

They actually did send out dev kits, though. So there are select developers. They essentially sent out, it looks like it's a Mac Mini that has, I think, like an eight 12 X chip in it, which is like the iPad pro chip or something. Okay. Um, and so that's what people are using to test development.

So they, when they did the announcement, they have like this signup sheet where, um, you have to like sign an NDA and put in a deposit and do all these things. But they've, they've facilitated some amount of testing for that. Okay.

So yeah, that's cool. So yeah, that's cool. So, yeah, that's awesome. Somehow, somewhere that got tested, presumably. Yeah. Right. Okay. Next thing is GCC 10.2 release, which is a bug fix release.

And they have a whole list on Bugzilla of all the different bugs that got fixed, a little over 90 of them. Any interesting ones you saw on here, Jason, that you wanted to call out? No, but the Reddit discussion is hilarious, at least the top-rated comments are. The nine stages of debugging your C++

code basically is in the Reddit discussion there that I won't spoil it for anyone. Yeah, it's definitely worth reading through some of the comments here, though. Yeah, sorry. One thing I noticed was it and I haven't read these kind of feature lists in a long time. But it looks like 10.2 rounds out the concept support in GCC. Oh, is that right? That's what I'm seeing when I look at the feature list. It just says like 10.2.

There's a little parenthetical comment, so it's maybe not fully complete, but it looks like it's mostly there, which is really cool. I mean, I remember hearing about concepts for a long time, so it's cool to see it kind of finally coming together. I know it's been like a controversial feature so not everyone's happy with with where it landed but i think it's great that it's at least landing and people will be able to get some benefit yeah there's at least five or six issues in here that are uh directly related to fixing concept support i've noticed in my own playing around that gcc's concept seems to be one tiny step ahead of Clang's concepts right now. So it's really cool. Yeah, to see them all coming in. Yeah. And in general, just quick shout out to

the GCC developers. I think they don't, they don't get as much kind of praise is kind of the LLVM, which is the more visible kind of compiler community these days, but they do a lot of tireless, wonderful work. And in the entirety of my VFX industry time, GCC has been the backbone for all the C compilation we do. So great job to all of them. That's awesome. It's probably also worth pointing out too, since we're talking about C++20 support. If you look at the compiler support for C++20 features, GCC is ahead of everyone else right now. It's honestly incredible on both sides.

It's always incredible to me how fast they get stuff out. I remember when C++14 was getting ratified, and Clang was like, oh yeah, we already implemented all that. It's like, wait, when? Yeah, that's a funny one, because I think Clang implemented all of that stuff, then submitted the proposals to the standard for a lot of C++14 related things. Yeah, it makes sense too. C++14 was, if I remember correctly, a much smaller release.

Yeah. Okay, and then the last thing we have is that C plus plus on C, uh, the videos are already becoming available on YouTube. I'm not sure if it's all of them, but it looks like they have about, uh, 50 videos, including 12 that are lightning talks.

Uh, so I'm not sure if that's everything. I don't remember how many tracks they had, but, uh, it's great to see them already available on youtube for those that did not uh attend the virtual conference and want to watch the content yeah if you look at the list of videos that are online phil i assuming phil's doing a lot of this work but i don't know the conference seems to be posting about four a day

five a day depending on the day so it should they should be done pretty quick i would think yeah i did start watching one of these videos uh i need to finish it but uh sybrand did one where uh they made a compiler uh for try not to curse on the show but for uh you can say bf yeah bf which uh looked very interesting and uh it was impressive to do that in a conference talk yeah i also have two or three of them open now and like partially completed i started watching matt godbolt's talk on correct api construction last night but then i got tired not because. Um, not because of the talk, not because of the talk. It was just very late. Um, uh, yeah, it's, there looks like a lot of fantastic talks and it's, it's really cool that everyone's able to attend these things now. Yeah. Yeah. I feel like a little

bit of mixed feelings about this. Clearly the conferences are much more accessible now. And at the same time, it's just a completely different experience right yeah i think i'm i definitely miss being able to go and just meet people and say hi um like uh you know that i remember the last c++ conference i was at in person was c++ now uh which was two years ago in Aspen, I think. And like, yeah, that one's particularly like intimate. So you just get to have a lot of these like fantastic conversations with people in the community. So I do miss that part. Yeah.

Okay. Well, Josh, could we start off by you telling us a little bit about what exactly you do at Netflix? And I know in your bio, you say you work on the studio engineering groups. That's the actual like production of some of Netflix's original shows, I'm guessing, and not, you know, the back end of Netflix hosting and streaming all the content. Yeah, I don't work on kind of like, you know, keeping the website alive or anything. Luckily, I am not entrusted with such great responsibility. So if your

streams ever go down, it's not my fault. But they actually have the team that works on that stuff is brilliant. And anytime they show us stuff, it kind of blows my mind. So yeah, I work in the studio engineering group, which is very similar

to what I did at Pixar and Disney. And so what I work on in particular is underlying libraries that take care of data pipeline transport issues. So one example is artists tend to use many what are called DCCs or digital content creation applications. So you can think of these as like Maya or Photoshop or Houdini, right? And so kind of shuffling the data from one place to another, there's a lot of libraries involved in that. And so I tend to work on the optimization of like export and import utilities across those different tools.

And then within the tools, a lot of times there's interesting data transformations you want to do, and there's typically underlying low-level libraries that take care of that and make sure it's performing. The reason I call out underlying libraries is typically the way these APIs are surfaced is in Python. So there's developers that typically work in Python, and they write a lot of scripts for kind of automating different tasks for artists but that python calls into c and c++ so there's those are

kind of the two core languages for animation and visual effects is python and c++ i feel like if you don't mind me and you take one step back maybe for our listeners who aren't familiar. And forgive me if you did say this. But you're referring to the fact that Netflix is a movie studio at this point, right? I mean, there is a part of Netflix that produces original content. But that's who you're working with, right? You're working with the artist on the original content stuff. This isn't pipelines of processing compressed video or something. No,

no. There are very cool teams of very smart people who I actually talked to pretty recently who do work on like compressing video and all that stuff. And like new algorithms for compressing video, which is kind of mind blowing. But yeah, we're in the content creation business. I'm not as close kind of to the production side as I would be at Disney or Pixar

which are just kind of smaller operations so you're just kind of all in one building but the same kind of concept, building tools that enable creatives to create That's a weird statement, I'm sorry that you just said that Disney and Pixar are smaller than Netflix when it comes to this

kind of thing. Yeah, Disney or Pixar when I say Disney, I mean Disney Animation. So it's just the animation studio. And smaller in the sense that they're all kind of co-located into one place. Whereas Netflix is much more of kind of a, at least within our org, is distributed across a few places. Though we're all distributed now. So what does anything mean anymore?

Yeah. I almost said something, but I decided not to bring that up. Before we get into some of the more technical stuff, can you tell us some of the original content projects that you've been involved with? Sure, sure. So I can't say anything for netflix i can't talk about any of those titles but there are some movies that have come out that i can talk

about so for pixar i was credited on coco which is my favorite movie to work on there and cars 3 which if you haven't seen cars 3 nobody gives it a chance it's actually like awesome and people should watch cars 3 it is really good actually yeah, and people should watch Cars 3. It is really good, actually. I watched it on an airplane, which I know is not the full movie-watching experience, but it was still really good. But Cars 3 is legit underrated. And then when I was at

Disney Animation, I worked on Frozen 2 and Wreck-It Ralph 2. And those were both fantastic projects. So those are kind of like my four public credits. I've worked on a few other films at Pixar and at Disney. They were just, I had only been there for a little bit of time. So you're usually credited once you've been there for a little while. So if I had watched the 10,000 names at the

end of the movie scrolling past, yours was in that? Yeah. Yeah. Like Frozen 2. I got Prime, Real Estate, and Frozen 2 on the credits as well. Like I was in there yeah yeah like frozen 2 i got prime real estate and frozen 2 on the credits as well like i was in the middle which is like it was like it was it was a real come up i was i was excited that's awesome very cool how many times have you seen frozen 2 rob uh just the ones i think my daughter's probably watched it two or three times though yeah i was gonna say since frozen one that got seen a lot in this house though yeah when i was when i was like younger and i had to like babysit my cousins um one of them was obsessed with finding nemo i think i watched finding nemo like 57 times over it was like that was like legit all they wanted to do they were just like they'd show up to the house and be like all right put on finding nemo like all right yes you guys should know how it

ends by now but all right that's awesome yeah so you mentioned this in your bio that you started with a programming language or you have programming language background. How did you make this move from programming languages into all this VFX stuff? Yeah, that's a good question. So I'll just give kind of a little quick history that kind of probably explains it, and if not, we can just follow up with other questions. So after I was finishing my undergraduate degree,

I had been working at a spectrum analysis company. So they build like testing equipment for cell phone towers, like embedded devices, all that kind of stuff. That sounds like fun.

Yeah, it was really interesting. Surprisingly, they were still able to use like pretty advanced libraries on that stuff. Like we were using Qt 5 at the time, which was like pretty cutting edge. This was like back were using cute five at the time which was like pretty cutting edge this was like back in like 2012 wow but i was like i wanted to do something

kind of a little more um i guess i could say just like kind of uh obscure and do some more kind of exploration about like what i was excited about in computing at the time so i started doing some graduate work in programming languages and at the time this was in like DSLs and Haskell and like type theory and kind of all those things. I don't know if you've heard of automated theorem provers. It's come up on the channel a couple of times. People talking about being able to prove the correctness of software. Yeah. So I was really interested in kind of very, I'd say, out there stuff in terms of languages.

And only out there in the concept of how much kind of mainstream programmers would really be able to use it day to day. And then kind of after about a year of that, I was pretty homesick. And I was missing kind of some of the practicality I'd been feeling at my job. And I got an offer from Pixar and Pixar was in the Bay Area and I was originally from the Bay Area. So it was a pretty natural fit for me to kind of go home and start working there. And so that's, that's all that really happened in the transition there as I was homesick and I had a large background in working with C++. Like that's kind of what my, my original like interest in programming all started in C++, that's kind of what my original interest

in programming all started in C++. Everybody else, I wanted to program video games, right? I mean, that's like a lot of C++ programmers, like, how did you get started? It's like, I wanted to make games. And so that's how I

fit at Pixar, because obviously everything's in C++ and Python for that world. So I had the right skill set, and then once I started working there, I just fell in love with the task because visual effects is one of those really cool industries where you're getting to work on these really neat properties, but also you have these interesting performance problems.

The performance constraints are really significant in visual effects, much like games but in a different way and so that was super compelling and i just kind of never looked back awesome very cool has your programming language background been been useful in the visual effects industry i i think it's like it's a double edged sword right like so like when you've spent enough time in like programming language, just like any other thing, it's like, I know what's out there. And it's like, sometimes I get frustrated,

particularly when I'm having to use Python, I get a little frustrated just because some of the tooling I find lacking for kind of ensuring software correctness. But on the whole, I think it's been great. And the way it kind of manifests for me is just really, really heavy leaning on using great tools, right? Like in C++ land, we have this fantastic suite of tools, like all the sanitizers that

LDM and GCC put out, right? And those things are just incredible. And I think really it's just been leaning into using all the tools at our disposal to make C++ easier to write more efficiently and more correctly. I'm on board with that. Do you then find yourself when you do need to use Python, like trying to use the gradually typed Python or some of these other things to try to give you another layer of correctness. So I would like to do that. Um,

the visual effects industry, uh, you had some buddies of mine, Carl and David Adler on weeks ago, and they mentioned the visual effects platform, right? We are just now moving on to Python three and the visual effects platform. You mean away from the thing that was deprecated and ended of life like 10 years ago, you're just now getting rid of?

You know, I'm not going to confirm or deny the deprecation status of Python 2, but yeah, so we're just now moving there, right? And so like a lot of the great tooling that has been made available in Python that would have made me complain less is almost all tied to Python 3. Right. available in Python that would have made me complain less is almost all tied to Python 3. There's two or three different really impressive looking type checkers. There's MyPy, there's Pyre, and I think there's one more that I'm forgetting for Python, but they're all Python 3.5 minimum, Python 3.4 minimum. So you kind of hit a wall. You're like, oh, cool, a new Python tool. And then inevitably it would always be like, you can't use it. So, but now you are moving into a world where you might be able to start using some of these things.

Yeah. Yeah. And, and at Netflix, like it's much more clean slate. Like it's a pretty new org. Right. So we've been able to kind of start in a, in better place with with the tooling on that stuff right but it's like for for disney and pixar you know they have code bases that are 20 years old by now i remember when i first got to pixar there was a math library um written by like um a programmer that had been there a long time and it was literally older than me

and i was just like okay all right let's uh what are some of the libraries you've spent the most time working on throughout your career in vfx yeah yeah so the biggest project i worked on i think that i was working on even once i was at disney but it started when i was at pix, is a project called USD, which stands for Universal Scene Description. So this is... Not the U.S. dollar. Not the U.S. dollar, which is an unfortunate acronym clashing,

but what are you going to do? Right. So USD is many things, but you can think of it kind of from a 10,000-foot perspective as a file format, an underlying set of libraries to manipulate that file format that acts as an interchange between those DCC applications I mentioned, right? So I could spit something out of Maya and pull it into Houdini, or I can spit something out of, you know, Katana and pull it into Maya. But, like, the interesting capability that USD gives you is it has this notion of composition.

So the idea is that assets are going to be getting created from many, many places. And a lot of times we want to be able to do these groupings and granular non-destructive updates. So an example would be, I have three or four models. So these are geometry files coming out as.usd files coming out of modelers. And I want to go ahead and pull them into a scene, and then I want to translate them into different places. So I can use these notions of composition to say I want to pull in this, this, and this model,

and then I want to apply a transform on it. And this transform I'm applying on it is a non-destructive edit. And what USD does is it knows how to compose those based on kind of the grammar that it defines for composition ordering. And it gives you, like if you're a renderer or whatever, here's kind of your final resultant scene. So it's saying like, I know how to do all this fast composition logic. And it allows you to have these very granular files where it's like, this is my geometry file. This is like my shading file. This is my sets file.

And what this enables is many artists can work on the same shot at the same time. And so it really enables kind of parallelism across departments, which is really, really cool. This is a project you're continuing to work on. You've worked on for a long time also. I haven't worked on it much in the last year. I've been pretty busy since I started at Netflix. Like we definitely use USD, like we're definitely interested in USD. But it was something I definitely worked on from like 2015 to 2019 pretty heavily.

I feel like we get a surprising number. Well, perhaps it's just the nature of how we learn about people for the show. But a surprising number. Well, perhaps it's just the nature of how we learn about people for the show, but a surprising number of people who have been paid or have professionally worked on open source projects throughout their career. Yeah, it's actually super fortunate, and I'm really, really grateful. Not a lot of people get to work on open source as a job, and so that was such a unique opportunity and just seeing what it takes

to really put a C++ library out there in a way that works across all these platforms reliably. Like just like, you know, the build complications you deal with is like kind of immense. And then you get into things like platform differentiation. Like it's unfortunate that a lot of these libraries kind of duplicate the work of having and then you get into things like platform differentiation. It's unfortunate that a lot of these libraries kind of duplicate the work of having kind of this platform layer

that defines all these different things. It'd be great if more of that could merge into the standard, but it's really cool to kind of see how the sausage gets made there. So does this project also have to support that standard that the film industry uses? Yeah. I can't remember what it's called. The VFX platform, yeah.

So, yeah, it supports it. USD, last time I checked, compiles on Windows, Mac, Linux, like kind of your standard things. The standard operating system for visual effects is usually like Red Hat or CentOS. Okay. So that's kind of like the standard platform they use, The standard operating system for visual effects is usually like Red Hat or CentOS. So that's kind of like the standard platform they use, but it does work on Mac and on Linux. And it's C++14, last time I checked. And then it should just essentially work with all those libraries.

It interops with a few things like Boost and OpenVDB, so it has pretty good compatibility across those. But I know there has been interesting challenges with Boost. If the Boost version is new enough and you're on Windows and things like that, which isn't that surprising. Whenever I've had to build Boost on Windows, it's been a challenge. Or really, when I build most C++ software on Windows, it's usually a challenge. Or really, when I build most C++ software on Windows, it's usually a challenge. But I will say, I've been doing a lot of development on Windows lately in my free time,

and the VC package folks are doing a fantastic job. Things are so much better than they used to be. It's great. I can't remember if we've had someone specifically from the... We have had someone from the VC package team on. I think so. It's been a while, but we have had someone from the VC package team on I think so it's been a while but we have yeah Billy O'Neill was

on pretty recently right oh that's right yeah he just switched over to VC package I think from working on kind of more the compiler team or the he was on loan but I think a recent tweet from him said that it's now his full time move oh okay

I think it's honestly like a super valuable thing for the C++ community to have this. I kind of hope VC package gets more and more traction and we can all just kind of agree on a package manager. Because I think outside of language features, it's one of the biggest problems in C++. It's just like every time I go to build something, it's some bespoke setup from someone. Yeah. Yeah's just like, every time I go to build something, it's some bespoke setup from someone. Yeah. Yeah. I mean, I've settled on Conan myself and that's, you know, I'm glad though, that we have at the very least a few options that seem to have a standard set

of packages available, right? Like we can pretty much assume boost and lib format and whatever's going to be available in every direction and things are definitely getting better lib format uh correct me if i'm wrong this is just a curiosity is that the one that's getting merged into the standard it is but specifically only the form this is sounds a little silly of it it's a subset it's only the formatting subset so lib format also has its own like printf and its own io stuff so if you want to use lib format with io streams or whatever then you would have to do like format and then pipe the insert that format into cout basically like so it kind of ends up as this

weird hodgepodge but it's easy to to write helper routines that make it look pretty. Okay. Yeah, that makes sense. It's actually one correction I might make. I haven't checked, but I'm actually not sure if, I'm going to check right now, if USD is actually technically C++11. Oh, it's opposed to 14.

So for a long time, everywhere I've worked, when we've been kind of forced into C++11 compliance, we get C++14 by just having a header that adds the few things we really care about out of it, like make unique and stuff like that, because it was always easy enough to write make unique. And that was always the one big glaring omission for me that was that was the one thing i missed yeah i strongly feel like c++ 14 is a bug fix to c++ 11 and you should just skip 11 if you can yeah yeah um and namely

a big thing that we noticed back when i was at pixar is 14 the compiler support especially in GCC was just a lot better in terms of compliance but also performance namely like I remember pretty early on the std regex was like implemented but like the performance was really really rough so I remember someone moved a thing

from boost over to std regex and like we have these like nightly performance tests kind of like everyone else, and they just tanked. We just reverted back to Boost Regex and it was fine. I'm not surprised at all. That might still be the case. It got better. I'll give the GCC developers credit.

It got a lot better. But, I mean, if Boost is kind of your performance benchmark, not a great place. Well, as long as they're still talking about USD and I get that you haven't worked on it in the last year, but you did spend a lot of time on it. And you said it's this universal screen description. I just, I just have this like mental image of like when I'm using open office and I say save this document as a word format, it's like, oh, this is going to work,

but you're going to be missing some features, right? Like this always happens when we're transferring between file formats. And you're talking about how like all the different 3D software uses this thing. Do you have the same thing happen when you go to export from the native format to this, that it's like, hold on,

there's a few things that I can't do? This is actually like a really great question there's a few things that I can't do? This is actually a really great question. So a lot of times what happens, I'll just give an example. When I was at Disney, a lot of times we had a lot of specific stuff going on in Maya that was pretty specific kind of to our plugins or like just custom in-house software we were writing. And what USD kind of allows is like, there's what I like to call this kind of escape hatch of like metadata. Metadata can be these untyped dictionaries that you can just kind of jam into a file. So essentially any time that we were like,

oh, we don't really know how to encode this like properly in the scene in a way that transports, which didn't happen too often, but when we did, we were like, oh, we could just put it in metadata. And we would create a little wrapper API for accessing that metadata, but that would essentially get us where we needed to go in terms of encoding any kind of application-specific things. So there was always kind of a way around that. It sounds like that then becomes a really hard design decision

when you go, okay, is this important enough that we need to add a new feature to USD versus jamming it into the escape hatch? Luckily for me, there were much smarter, more experienced people who kind of make those decisions at the global level. I was very much like the young pup in the USD group of very experienced C++ developers when I first started there. Namely, Sebastian Gracia, everyone calls him Spiff. He's been at Pixar forever

and he's kind of like the lead, one of the lead designers of USD. And he's just super thoughtful about like what really needs to be included and what doesn't. In the same kind of way that the standards committee is like very thoughtful

about the implications of changes to kind of the C++ standard. So luckily there's really good kind of arbiters working on that. Awesome. Okay. Today's sponsor is the PVS Studio team. The company develops the PVS Studio Static Code Analyzer designed to detect errors in the code of programs written in C, C++, C Sharp, and Java. Recently, the team has released a new analyzer version.

In addition to working under Windows, the C Sharp part of the analyzer can also operate under Linux and macOS. However, for C++ programmers, it will be much more interesting to find out that now you can experiment with the analyzer in online mode on the godbolt.org website. The project is called Compiler Explorer and lets you easily try various compilers and code analyzers. This is an indispensable tool for studying the capabilities of compilers. Besides that, it's a handy assistant when it comes to demonstration of code examples. You'll find all the links in the description of this episode. Well, if you haven't worked on USD too much in the past year on Netflix,

what have you been working on? So a lot of NDA things, but essentially, like I've been working on in the same space as USD communication tools between applications. Right. So it's like, how do we effectively communicate? But you can imagine in a more distributed world.

So like things that kind of modern, modern computing companies, right. He's a lot more distributed across like cloud services. Right. So then it's like, essentially like, computing companies right is a lot more distributed across like cloud services right so then it's like essentially like how do you move kind of this kind of work into the cloud is and this is a challenge i think that a lot of companies will be facing just because more and more kind of this scalable computing is becoming a question that that places with on-premises

computing have to at least think about right and so it's how do you take this infrastructure and make it work in kind of this new environment that has like these elastic computing clusters that are all accessed over the internet. So I think it's building services that are scalable, but not just scalable with respect to compute, but also scalable with respect to kind of like network latency and things like that and being like fault and failure tolerant with respect to networks so i think that's that's been a really interesting

challenge for me because coming from those previous environments we didn't really have to care about the network at all right like in in a typical visual effects house like you're all on like a shared file system so like i guess you could say it's like there's a network in terms of nfs but not not like a traditional network as you would think of it so i saw this uh press release here from netflix tech tech blog that they are working uh on a community visual effects community with the net fx platform but this this whole thing is it's it's very light on details not being someone in the industry. I'm just curious if this is something you're involved in with your special effects work at Netflix

or anything that you can talk about. Yeah, so NetFX is adjacent to my group. So I know people in the NetFX group, but I don't directly work on it, so I can't really comment on their work. But definitely a lot of the core technology we're developing will be shared with the NetFX group. And they're a bit more kind of customer-facing,

customer in the sense of kind of artists and shows and things like that, whereas we're a bit more insulated. We build the layer below them of kind of core technology okay so for people interested it sounds like this is kind of a watch this space and see what comes up over the next few months kind of thing yeah yeah definitely definitely watch this space um i think uh actually doubling back to your to your kind of previous question there was one thing i i thought was would possibly be interesting to delve into um sure that you two might have thoughts on um which is kind of the performance profile in vfx

and like what performance kind of means in visual effects okay uh so i think like this is something c++ does like incredibly well right c++ allows you to target uh performance targets in in a very straightforward way i think chandler caruth said this a long time ago and i've always loved this quote he said like c++ doesn't give you performance it gives you like the ability to understand your performance right or like it gives you kind of like this like direct mapping. I may be like muffing the quote, but that was the gist of it if I remember correctly. And it was always interesting like when I hopped into VFX because to give you context,

a visual effects artist works on a workstation that has about 200 gigs of memory, dual socket, anywhere from like 30 to 60 cores, cores right and there's like big quadro graphics cards right so those are like kind of like the professional line of nvidia cards i don't know much about gpus so i can't say much about those but the performance profile of like those individual machines is really interesting because you have to run very efficiently on those but you also have to run a lot of these same processes on like a traditional render farm, right?

Which is like just thousands and thousands, like Carl was mentioning, thousands of cores running in like a data center. And a lot of times you need to kind of very much tune for both of those scenarios. I remember when I first, this is when I found out about performance and kind of got my butt kicked really early on at Pixar. So I was given a task that's like, essentially, you have

all these kind of per frame geometry files coming out of this effects application, which does like volume simulations and things like that. And it kind of for every second or every like frame in the simulation is spitting out like a per frame file, which is like a bunch of geometry. And you need to run something that can process every single one of these files in parallel and kind of generate what's called like an aggregate topology, which is like, this is all the geometry you might see. Because this manifest allows people to view it a lot more efficiently if they know everything they're going to be looking for. And so I was like, okay, cool. I know C++. I know what I'm doing. I was like pretty fresh out of college, didn't know what I was doing, but I was like, I got this. And so I go to write it and it's like, I do it on my little tests and I'm obviously using these small dummy test files because of course I was, and it runs fast. I'm like,

this is great. Submit it. And then like, it was like a week later and artists was like, Hey, my, my simulation export is taking like 45 minutes. And I was like,

Oh no, what's what's happening. And I go look at it. And for like a five or 10 second like export of like frame time, it was about 60 gigabytes of data. And so I had vastly underestimated the cost of doing this. And quickly, I ratcheted up my parallel programming skills because I was, of course, also doing this in a single threaded pass because I was like, it'll be fast enough.

Who needs multi-threading here? And of course, I was, of course, also doing this in a single-threaded pass because I was like, that'll be fast enough. Who needs multi-threading here? And, of course, I was wrong. So that was my quick introduction to the scale of data working in visual effects, but also the need to actually exploit these many, many core machines. And for that, in visual effects, the tool that we go to, which may be going away given the current standards trend is uh intel's tbb um and that's that's actually been like one of my favorite tools um is intel's tbb is just like especially for like basic like parallel four and parallel reduce and things like

that like they just expose like a really simple way of doing it like you just pass a lambda it does the thing and it's for like basic use cases it just kind of works um so that's that's been great but i don't know if the new parallel standards track i haven't followed it at all so you guys can let me know but i assume it's going to be replacing at least like parallel four and kind of the basic pieces there yeah i mean parallel four each was it's in c++ 17 right okay but Okay. But I just try to not, you know, sound optimistic, I guess I should say. Visual Studio has support for it. GCC has support for it.

Clang standard library still does not. So it's one of these things that's been super lagging. Like after the compiler said that, oh, yeah, we have full C++ 17 support. It's like a big asterisk that says, I mean, if you don't look at the standard library, so, uh,

but by the time that the, the visual effects, uh, standardization tools that you guys use catch up to C++ 17, you will have it. Hey man, in 2021, we were going to be right there on C++ 17

on the bleeding edge of technology. Well, hopefully your compilers that you need to rely on will have it by then. It's always funny to me because, yeah, visual effects goes at a bit of a slower cadence in terms of adopting technology. But when I first

worked at an embedded company, which it was very traditional, brown buildingicles like you know kind of your standard fare and like they were so bleeding edge with the c++ they used like we're on like c++ 11 in like 2011 or 2012 like right it was like not even fully implemented across the board and then like we're on c++ 14 as soon as like the Clang piece dropped. And this is running it on embedded ARM devices. And so it's just funny. I think a lot of times the size of your team determines how nimble you can be on those things.

And we were a pretty small group, so we were just able to move those code bases over pretty easily. Yeah, and now you're an industry-wide standard that you have to deal with yeah and you know carl and david touched on this and honestly david knows kind of the problem a bit better than i do but it's a lot of it is this abi issue right where it's like right you have to have like a lot of times these libraries would get released you know just as so files or dlls or whatever that need to interrupt inside these packages and it's like if there's any kind of like abi breakage across those you're just kind of hosed so then it's like everyone has to move in lockstep with like the same compiler the same basic settings and and all those kinds of things which which definitely makes it tough like what ends up

happening right is like most studios end up building kind of like a sandbox environment because like you'll have multiple shows right and a lot of times shows will lock to kind of different versions of things and so like you'll essentially have like you may have a different version of boost building in like different shows but you essentially have to like rebuild the world for that version so i mean i think the and and it can get pretty rough in terms of like compile time and stuff because again like you're kind of rebuilding the world i remember like the longest compile time i've ever seen was like three hours give or take and this was on like this was on one of those like big 30 40 core

machines so this is like yeah it can get it can get pretty bad. Um, but you know, what are you going to do when you're locked into a vendor? You just don't have much say over that. Right. I've, I've got this question, uh, that's kind of been forming and maybe I'm going out on a limb a little bit here, but you, you have your programming languages background and you've talked about how you want to rely on the tools and make everything as good as possible, correct as possible. And you're talking about these USD and what am I trying to say? Like a scene transformation, pipelining tools, right? So I'm imagining, okay, that a considerable portion of your work is like, this has to be exact. This has to be perfect.

When we translate these scenes, when we do geometry transformations, whatever, this is all perfect. And then it reaches a point, right? When it actually goes to be rendered and displayed by whoever, that accuracy is not the important thing anymore. It needs to look good and it needs to render quickly. And I'm just wondering, like, do you, do you have to deal with like, is that, does that border actually exist or am I just making it up?

Do you have to deal with it and does it drive you crazy? Yeah. I mean, there's definitely to your, to your initial question or part of it, like there's definitely this, it has to look good. It has to look right. but it kind of like the stage of the pipeline i've typically existed in you can think of it as a lossless transformation on most of the stuff i work on when you get everything we're gonna have to be exact yeah when you get to the stage of like i mean exact like we work with a lot of floating point data so nobody's gonna be exact but um and i actually have a quick funny bug story after this for that but on like the rendering side to your point there is an amount of noise that kind of david and carl got into a couple weeks

ago right where they have these like really cool tools called like uh i think they're called like perceptual diff and things like that which is like it can do an image diff but it has like this tolerance level so it's like does this actually perceive to be the same image even if like you know two pixels are different so it has it has a kind of intelligent way of understanding this is basically the same image there's no substantive like transformation that's like really like making this look improper in some way um the funny bug that i ran into so this is uh there's this library called open exr okay yeah yeah so open exr has their own half floating point type and uh it's like so it's like a 16-bit 16-bit yeah there's some talk about standardizing 16-bit floats actually yeah i think i think a lot

of people in games have maybe talked about it for like the space saving and stuff like that um right but so like they have a constructor and the constructor takes a float right and obviously it halves the precision and it uh i was running into this case where like all my data was coming out wrong and what i found out was the constructor was non-explicit but it like took me so there was a function that accepted a half but i didn't realize that was the function getting invoked because half had an implicit constructor from float. I thought I saw a function that took float. And this was like, I was like, you know, you've seen that picture from It's Always Sunny where the guy looks like crazy and he's pointing at all the drawings of the wall.

I was at that point of looking at my code and I was like, what is happening like am i crazy like maybe maybe i'm actually crazy and like i should just like leave um because and like when i finally figured it out that was one of my few like c++ table flip moments and i was just like ah make your constructors explicit. Why do you hate me? And so, yeah, opening XR will always hold like a special place in my heart for that moment. That's awesome. And that's great too, because I mean, if it's a built-in conversion, you can at least ask the compiler to warn on it.

But since it was a user-provided conversion, I can't think of any way that you could have even gotten a warning that you were doing this. That if I was going to ask the standards committee for something that does remind me, if you're listening standards committee, I want,

I want strong type deaths, please. Strong, strong type deaths. Like, uh, look at new type in rust or Haskell. That's what I want.

If you want me to write the proposal, I'll do it. I need it i talked to someone about that was it on air i don't remember now i don't remember there there there have been a strong type def proposals i believe for as long as c++ has been a standard but they always get abandoned because no one can agree on what exactly it would mean. That's, that's my understanding.

I think you're correct. Cause I've looked around before and I've seen them. Um, it's, it can be emulated pretty easily. I remember someone showed me this at like, I think I was actually at like a Haskell conference and then someone was just like,

look at this cool C library trick I did. So I don't know how it kind of came about, but essentially it's like, if you have a struct with a single element in it, the compiler is smart enough to treat it as if you were just holding that single element. So it knows how to align anything there.

And I think all you have to do is have an explicit constructor on it and then you kind of get what you want in terms of those conversion issues. I think that the problem and why it's never been standardized is which operations does it have and how do you enable or disable said operations?

Yeah, I think it's, you know, this actually touches on like a really valuable point I was hoping to make, which is that like C++ solves like a really hard problem. And I think the PL community doesn't give it enough credit for what it does. It's super fun for them to dunk on C++, right? I get it.

C++ is super complicated. We all know that. But they have the really hard problem of maintaining backwards compatibility, right? So anytime they make these decisions in the standards committee, they're like, I can break this code from like 1992 like that code needs to that code needs to keep working and so like every decision they make has these like deep implications and then it's like when you don't have a module system or kind of like any way to properly separate these compilation units they're just left in like a really difficult place like the fact that they got lambdas into the language is kind of incredible right like i so i just hats off to the committee like i don't agree with like everything a really difficult place. Like the fact that they got lambdas into the language is kind of incredible.

Right. Like I, so I just hats off to the committee. Like I don't agree with like everything that happens and like, yeah, the language frustrates me sometimes, but like they're solving a really hard problem. And so like,

they don't, they don't get enough credit for that. Like there's things that could definitely be more optimal. I wish people supposed to have like real proper algebraic data types, but like, again, they're having to work in a way that doesn't break old stuff. Yeah, so to that point, you said your industry is just now moving to Python 3.

And that's because Python 3 to Python 2 was a break. If C++ had a break, forget it. How long would it take you to cross that break? I think that's what's always been interesting to me, is people talk about, like, oh, let's just fork the language. It's like if you want to fork, just write Rust. Rust is a good, clean room implementation. It's like the reason we're not forking is because we want all that old stuff to keep working. Right.

And C++, to be clear, has gotten a lot better in the last five or ten years, however long it's been now since C++ OX came out. It's gotten much, much better. But yeah, if you have a clean room thing and you have maybe big security concerns or anything like that,

Rust is a great option for that. But a lot of us have these big code bases we need to keep working with or interop with and like c++ gives us a great option to do that so yeah yeah a fork is just not like a legitimate option in my mind no i think herb has mentioned and this is a long time ago so i don't know how he feels about it now but herb has mentioned the notion of like flags to say like i'm in kind of new c++ mode they can compile things like maybe with some of the c backwards compatibility gotchas out of there right um and maybe that's a way forward but i think again you probably need to get to modules before you can really have proper kind of compilation modes that way they can all kind of separately compile to this intermediate representation.

I kind of, sorry. No, no, no, go ahead. I kind of personally feel like by the time you've enabled a considerable portion of your compiler's warnings and run it through Clang Tidy or something similar and are willing to actually listen to those warnings, that we've pretty much done that.

We've pretty much, I mean, we've effectively forked the language without breaking backward compatibility. Just only use these things that don't generate warnings now. I agree. I agree. And like, yeah, there's issues of like spurious warnings making too much noise. Like that always happens in these kind of systems. And it can be a little frustrating. What I've come to find in projects though is once I've set up a proper kind of like ignore list to like turn off warnings that don't really matter to me. This happens in Python too with Pylon and all these kinds of tools that have frivolous warnings I don't really care about.

But once you've kind of set up your infrastructure, it hasn't really been a problem for me. And it's mostly caught issues that I was glad that it caught. Like sure, sometimes you're like, oh, it's nothing. Leave me alone. And I think a lot of that comes down to the ergonomics of properly setting up a development environment where you can develop without being bothered and then on submission to CI actually do all those checks is usually how I set it up.

Where it's like, let me make my mistakes and write this kind of crappy first pass code and then you can yell at me later, but before it actually, before it actually gets like merged in. Right. Right. Okay. Josh,

it's been great talking to you today. Uh, where can people find you online? Cool. So I'm super funk with a C on Twitter. If people want to see me talk about programming and mostly only programming. Um, and I'm also super funk on GitHub. So my, talk about programming and mostly only programming.

And I'm also superfunk on GitHub, so my website is just superfunk.github.io because I'm too cheap to pay for a domain, but I don't blog very often, so Twitter is the best place to get a hold of me if you want to talk to me. Okay. It's been great having you on today. Yeah, thank you guys so much for having me.

I really appreciate it. Thanks. Thanks so much for listening in as we chat about C++. We'd love to hear what you think of the podcast. Please let us know if we're discussing the stuff you're interested in or if you have a suggestion for a topic. We'd love to hear about that too. You can email all your thoughts to feedback at

cppcast.com. We'd also appreciate if you can like CppCast on Facebook and follow CppCast on Twitter. You can also follow me at Rob W. Irving and Jason at Lefticus on Twitter. We'd Thank you.