CppCast - Hyperion Renderer

Episode Date: July 9, 2020

Rob and Jason are joined by Yining Karl Li and David Adler from Disney Animation Studios. They first discuss C++ conformance changes in Visual Studio 2019. Then Karl and David talk about the Hyperion Renderer and more. News C++20 Features and Fixes in VS 2019 16.1 through 16.6 JSExport: C++ in the browser made easy Magnum 2020.06 released Cpp On Sea: Full schedule, Nico Josuttis Keynote and Remo as a platform Links Disney's Hyperion Renderer Sorted Deferred Shading for Production Path Tracing The Design and Evolution of Disney’s Hyperion Renderer VFX Reference Platform Sponsors Clang Power Tools

Episode 255 of CppCast with guests Carl Lee and David Adler recorded July 7th, 2020. Get ahead of bugs by using LLVM static analyzers and CPP core guidelines checks from the comfort of your IDE. Start today at clangpowertools.com. In this episode, we discuss C++ conformance improvements in Visual Studio. Then we talk to Carl and David from Disney Animation Studios. They tell us about the Hyperion renderer and how it differs from traditional renderers. Welcome to episode 255 of CppCast, the first podcast for C++ developers by C++ developers. I'm your host, Rob Irving, joined by my co-host, Jason Turner. Jason, how are you doing today?

I'm all right, Rob. How are you doing? Doing okay. I don't think I really have too much news to share myself. Anything from you? Nope. No? Okay. We can just get right into it then. Well, it's time for episode to read a piece of feedback. This week, we got an email from Eric saying, thank you very much for the awesome podcast.

I'd love to hear a podcast about the C++ aspects of PyTorch and TensorFlow, which are the most important machine learning libraries today, and they are written in c++ uh and yeah i certainly you know hear little bits and pieces about those libraries and we probably mentioned them a handful of times but we've definitely never done an episode about them um and we should i have no idea who would be a good person to talk to from either of those communities but we should try to find someone neither do i so if you're listening and you know someone involved in the C++ aspects of these AI libraries, reach out to us.

Yes, definitely. Or if you know someone who would be a particularly good speaker on either of these topics, definitely let us know. Okay, well, we'd love to hear your thoughts about the show. You can always reach out to us on Facebook, Twitter, or emails at feedback. cbcast.com and don't forget to leave us a review on itunes or subscribe on youtube joining us today are carl lee and david adler carl is a senior software engineer at walt disney animation studios where he works on disney's hyperion renderer as part of the disney animation rendering team his areas of interest cover all things production rendering, and he has contributed rendering technology to the films Zootopia, Moana, Olaf's Frozen Adventure, Ralph Breaks the Internet, Frozen 2,

and upcoming film Raya and the Last Dragon. Previously, he was at Cornell University, Pixar Animation Studios, DreamWorks Animation, and the University of Pennsylvania, where he studied something completely unrelated to anything he does now. And David Adler is a software engineer at Walt Disney Animation Studios, where he works on the Hyperion renderer and its machine learning denoiser. He has been at Disney since 1998. When not working from home, he's working on his home and automating it.

Welcome to the show. Thank you. It's great to be here. Yeah, thank you. So just out of curiosity, since you said you were previously at Pixar Animation Studios, Carl, I was just recently reading some stuff about the history of Pixar's acquisition into Disney. And just out of curiosity, I guess, for a foundation for this conversation, Disney Animation and Pixar are completely separate managed entities today?

Or is there overlap? Yeah, so the two studios are completely separated okay you know pixar is up in sort of the bay area uh disney animation is down here in la uh and yeah they're kind of sister studios that work on their own projects completely separately they have their own staff's own sort of executive leadership structures own separate creative teams. On sort of the technology level, there is a degree of collaboration and co-development and sharing, you know, ideas and tools and code and stuff. But for the most part, the studios operate kind of independently from each other. Okay. And David, something else stood out to me in your bio as well. A machine learning

denoiser. I just feel confused as to why that's necessary for something that was computer generated in the first place. I feel like denoisers come with an aspect of error from CMOS sensors or something like that. Yeah, the reason is Hyperion, like most path tracers these days, is a Monte Carlo simulation of all the light bouncing around in the scene. So there is inherently noise or randomness in the result that it produces. And the longer you render, the more that noise

diminishes. But if you don't want to render as long as it would take for the noise to become imperceptible, you run a denoiser on it after it's done rendering. I had no idea. That's really interesting, actually. I've seen some of those Monte Carlo ray tracers, renderers, but I didn't realize, huh, all right, thank you. Yeah, I think that's been one of the major shifts in the industry over the past 10 years is that kind of starting from, yeah, roughly 10 years ago to about right now, basically every single major animation studio and VFX studio has moved over from sort of more traditional Reyes style rasterization renders to Monte Carlo path tracing. Okay. And games are starting to move that

direction too. Interesting. So I've only played very, just a little bit with 3D rendering and animation way back in the day. And way back in the day, if you're using an open source ray tracer, and you did anti aliasing on the image, then you the anti aliasing used some sort of random generator. And I believe that if I'm remembering this all correctly, you all know the industry and stuff way better than I do, that you would end up with from frame to frame the potential for noise being added to the video because of the anti-aliasing or whatever effects had been applied until they're like oh wait you know you just use the same seed on each frame and then that problem goes away so just out of curiosity now i'm wondering if your denoiser has the potential

to effectively add some sort of frame-to-frame noise that would then be a problem. So it actually removes frame-to-frame noise also. Okay. It operates, to output one denoise frame, it reads in seven input frames, which means it's not only removing the spatial noise, it's also removing temporal noise. Okay. And as a machine learning denoiser, it's trained on thousands and thousands of examples of this is a noisy frame, and this is a very clean frame. And it learns to produce a clean frame from the noisy frame without us having to hard code exactly what the algorithm is for how it could do that. Awesome. Very cool. Thanks for the diversion.

Okay, well, we got a couple news articles to discuss uh both of you feel free to comment on any of these and we'll start talking more about the work you do on the hyperion renderer okay great okay so this first article we have is from the visual c++ blog post from stl about c++ 20 features and fixes in vs 2019 16.1 through 16.6. And yeah, they've been pretty busy over the past few months. So busy that they didn't have a chance to split this out into a few posts. And so they're just summarizing all the different C++ ISO-related changes to the compiler in this post.

There's a lot here. There's a lot of stuff yeah i don't know if there's anything specific you want to go over uh but they're starting to you know bring in more and more uh c++ 17 and 20 features um i don't know anything you want to mention jason uh it's funny you should ask because when i was reading this earlier today i said hey i definitely want to bring up that thing and i don't remember what it was so uh that's why i think they said they had concepts in already, right? Well, I saw that there's sort of continued progress on getting coroutines up and running. And for me personally, coroutines is a very exciting C++20 feature for a variety of reasons.

I feel like I really need to see some practical examples of coroutines. It's something I've had just a hard time wrapping my mind around for a long time. So I'm really looking forward to when the compilers support it well, and I can start playing with other people's examples, basically. Yeah, we can get into this a bit later, maybe, but there's a sort of a specific application to ray tracing that I think potentially is really interesting. Okay, let's link into that later.

Okay, next thing we have is JS export. C++ in the browser made easy, which I don't think I've heard of this before. It's called Chirp, I believe. And it's able to basically compile from C++ to JavaScript so that you could quickly get some C++ code onto the web and test it out. Kind of an interesting concept.

Yeah, I found myself a little confused because this is not compiling to WebAssembly. No, you can compile directly to JS. JavaScript. Yeah. But I think there was an option. You can output a combination of JavaScript or WebAssembly. And I think the WebAssembly version

was actually a little bit more capable. Yeah, that's what he says later. Because it has more C++ concepts that it can map or something like that. Which makes sense. Does this technology compiling to the web have any application for either of you? Not currently. We have a piece of our renderer that's actually a web server.

And we have a JavaScript application that runs within that. So potentially, I guess we could have done that in C++. But I don't know that it would have been worth doing that way. It's probably still easier in JavaScript when you're targeting the browser. What does the web server component of your renderer do? It's primarily a way to interactively monitor the render and see live stats, statistics on how the renderer is doing. Okay. It's just a really easy user interface, and then we can also do REST APIs if other programs want to connect in. That makes sense.

Yeah, it's actually pretty neat because we have a server farm consisting of thousands and thousands and thousands of cores, and for our artists, it's really useful for them to be able to send a job that runs across the entire farm and just get back a URL, and they can click on it inside of a browser anywhere in the studio and they can kind of monitor

exactly what's going on with their work. Can they then also observe like click and say oh this is the frame that was output from this web interface? Yeah you can actually see the live render sort of as it's updating. Neat. That's pretty cool. Okay this next one is something uh another library that i've never heard of but i'm wondering if our two guests might know more about it uh magnum 2020 06 was released and it's a geometry pipeline tool and it definitely seems like it's pretty relevant to uh you know the rendering community i'm not familiar with that one

no okay how about you carl i i have i'm not too familiar with it personally i've seen it sort of uh pop up on occasion inside of like sample research code bases so yeah i took a look at it last night and it seems to have a lot of good stuff in it yeah definitely i don't know anything you want to add to this one jason no i also had no idea what it was i ended up going back up to the main page to try to like get a little bit better understanding as to what i was looking at and that's where i left off yes yeah lightweight and modular c++ 1114 graphics middleware for games and data visualization so yeah if that's something you're interested in, this is another option. Okay, and then last thing,

we have another update for the upcoming CPP on C conference. And they added another keynote speaker, Nikolai Yosudis, who we had on recently. And they also are announcing the platform they're going to use because the conference has gone fully virtual, obviously, and they're using this new platform called Remo, I believe, which I have not heard of. But I looked into it a little bit, and it seems like it's an online video platform that's really trying to try to kind of recreate the conference experience as opposed to something like Zoom,

which is just a massive call. You could have a hundred people on and can't really have small group conversations, but this is kind of built for that, which I thought was interesting. Yeah. Like random tables of people hanging out, you can go chat with people and join the conversation. And yeah, so hopefully this goes well. I mean, because over the next year, uh, we'll probably see more and more conferences go virtual. So we'll have see more and more conferences go virtual so we'll have to see uh how cpnc does with this remote platform hopefully it does well i feel like we've come full circle in a way if you go way back to the uh you know like ultima online or something like that people were meeting and hanging out and then they would do like big

gatherings and have a wedding online or something and And that stuff started what in the early nineties with some of the earliest online environments. And now it's now that's now that's a business application effectively. Right now. I mean, that's kind of weird, but kind of cool also. So what the conference is soon,

I think it's in like two weeks, right? Um, let's just double check that real quick, because if you want to sign up, you should go sign up now. Yeah. July 15th through the 17th. Um, it will be in European time zones basically. So it'll be a little bit harder for those of us on this call to participate in a large part of it, but yeah. Okay. So I guess we briefly mentioned in both of your bios, what you work on,

but could one of you tell us a little bit more about what you work on at Disney animation studios and the Hyperion renderer? Yes. Carl. Sure. Yeah. So, you know, as, as you mentioned,

both of us are a part of the rendering team at Disney animation, which is responsible for sort of a whole suite of various tools built around production rendering. The biggest one and most noteworthy one being Disney's Hyperion renderer, which is our in-house production renderer that's responsible for basically producing every single pixel that you see in the final film. So it takes in all of the input data from the rest of the studio, so all of the animation data, all of the geometry and models, all of the textures, the shader information, the lights, so on and so forth. And then it does an enormous Monte Carlo light transport simulation to sort of produce...

Well, I would say produce the final frame, but really the output of the renderer then goes into a compositing package, which then goes into a color grading suite and so on and so forth. But maybe the best analogy is that, I guess, in an analogy to a live action pipeline, it's the equivalent of the moment the photon hits the camera sensor. I'm actually, okay, I have no idea. I feel like I'm at a high risk of derailing this conversation constantly away from C++ and more towards the actual pipeline and stuff.

But I'm fascinated, since you just said this, that you, after rendering, it's because it's a perfect computer-generated image that you then have to do color correction. That's what you said, right? Yeah, that kind of thing. What? Why? They call it color correction, but it's not correction in the sense of it being scientifically correct. It's more like artistic adjustments. Okay, so it's a scientifically perfect image that came out and then the artists have their way with it? It's scientifically perfect in the sense that it honors the lights and the geometry and the materials that were given to it. But the artists work more or less on a shot by shot basis with

some elements of full sequence work to have continuity between shots, but they don't always have exactly perfect continuity. So one thing they can do is adjust scenes to match better with each other, but they'll also make larger scale adjustments, like even just to convey the mood a little bit differently than what was originally envisioned. Maybe they'll want to make lighting more dramatic, or maybe they even want to tone up a certain light and tone down another light, and they can do all of that in compositing. Yeah, see, five minutes ago, I would have just assumed that you would have had to have re-rendered the whole movie if the artist had decided they needed to change the lighting for the movie.

Yeah, it turns out that's slow. That is slow. How long does it take to, I don't know, render? Can you even give any kind of metric? What kind of scale are we talking about so to render a frame to final quality might take from an hour to you know maybe 12 hours 16 hours depending on the complexity of the frame and that's using the entire that's just for one frame which is 1 24th of a second of the movie right and a single run of the hyperion renderer produces just one frame yeah and that's

across the whole render farm that it takes an hour? Typically it's running on one machine with maybe eight cores. Okay. And so I actually run multiple jobs on the machine at a time. All right, sorry for the diversion. No problem. Well, I guess let's go into C++ a little bit more. What version of C++ are you using?

What IDEs do you use? So we're on C++ 14 right bit more. What version of C++ are you using? What IDEs do you use? So we're on C++ 14 right now is our standard. And we're somewhat constrained by a standard called the Visual Effects Platform, which is something that the visual effects industry uses

basically because we have a number of large third-party content creation applications, and they all need to live in the same software environment. And they all share some of the same open source components. And if we didn't have some kind of a standard, we would be in versionitis, where application A needs a particular version of a third-party tool, but then application B wants a different version, and you can't have them both

installed in the same environment. And the industry has been kind of struggling with that problem for a while before they came up with this visual effects standard um vfxplatform.com actually if anybody's curious so once that now that that's there we can look and say okay we're on the the 2019 version of the standard which means we're going to use gcc 6.3.1 and so on and that gives us good interoperability but it does somewhat limit like it's If we advance to a more higher version of the compiler, we run the risk of some things not being compatible with it. Okay. And when do you go about upgrading to a new version of the VFX platform? Do you just make the decision when you're starting on a new movie project?

That's a good point, because it's difficult to upgrade in the middle of a show. And we'll also look at what are the new versions of the content creation tools that the shows want to use requiring. And hopefully, they want to advance to versions of different products that are all on that same standard. Sometimes they don't, though, and we have to actually dynamically switch the environments. Whenever they launch this particular application, we remove from the environment whatever was in it before and then switch to a different version of the environment. But we prefer not to do that because it's harder to maintain. That's interesting. That reminds me of, I can't remember the name of the tool, but the Python tool that lets you just dynamically switch which version of Python you're using whenever you want to.

Yeah, virtualenv. Okay, yeah. Yeah, it's a similar idea to that. Hmm, okay. you want yeah virtual env okay yeah yeah it's a similar idea to that okay so uh we talked a little bit about um you know this monte carlo simulation can we maybe go into that a little bit more and how hyperion differs from other ray tracers from i don't know maybe a decade ago yeah sure um so the the well about a decade ago um most production renders were basically scanline rasterization renders, which is at a high level conceptually very similar to how your GPU is doing raster

graphics for video games and such. So it's basically you take a triangle and then you kind of splat the triangle onto your frame buffer, which is a grade of pixels. And then you do that for every single triangle inside of your entire frame. Then after you've done that, you take all of the little fragments that have been splatted to the pixel grid, and then you shade each one of them.

So Monte Carlo ray tracing kind of works almost the opposite from that. Instead of taking triangles and then splatting them onto a frame buffer, instead you're firing out rays from a camera, and then those rays, which are just lines in space, they hit whatever objects that are closest to the camera, and then the Monte Carlo ray tracing part of it is

at each point where a ray has hit an object, you calculate sort of a scattering direction based off of the surface materials and the textures and all that kind of stuff. And then that ray will go off in a new direction, which is basically randomly selected based off of the surface materials. And then it'll go and hit another object

and it'll continue bouncing around until it either goes into empty space or it hits a light source or it gets killed by some other metric and then sort of the the total contribution of that path essentially represents the flight path of a single photon into the camera and that contributes one photon's worth of information which is similar to a photon hitting you know a ccd on a camera in the real world. And then by repeating this process billions of times per frame, you eventually gather essentially

all of the photons that are useful that have flown through the entire scene to produce your final image. So Hyperion was one of the first production render to do this, but it was certainly one of the, it wasn't the first sort of production render to do this, but it was certainly one of the earlier ones in sort of this current generation of ray tracing production renderers. It first went into, I think development started in, David can't answer this better than me, but I think development started in 2011 or 2012, if I remember. It might have been. It was first used in Big Hero 6, which was released 2014.

Okay, okay. Yeah. And then where Hyperion kind of differs from a lot of sort of our sort of peer renderers that are out in the industry today, there's kind of two major areas that I think Hyperion has notable differences from its peers.

The first one is a more technical one. It's in its core architecture. So the Monte Carlo ray tracing problem is kind of trivially parallelizable because you can basically wrap a parallel for loop around every path that's being sent from each pixel. And you can just parallelize on a per-path basis and kind of go depth-first with each path through the whole world. And that's what most sort of production renders out there do. But Hyperion actually doesn't do that.

Hyperion has this sort of batched architecture where instead of wrapping the parallel 4 around all of the rays, and then tracing each ray depth first, instead it's sort of a breadth first render where it's tracing sort of the first bounce of all of these rays. And then it's doing a bunch of sorting and batching operations

so that it can extract additional coherence and memory from packetizing rays that are going to similar parts of the world and accessing similar chunks of data and then doing sort of batch operations on those. And we call that sort of the sort of deferred shading architecture, which enables us to sort of scale Hyperion to extremely complex scenes and allows us to execute extremely complex shading systems with a high degree of memory coherency. So that gives you a performance advantage and a cache friendliness kind of way?

Yes. Okay. And then sort of the second key difference isn't really a technical one. It's more of almost a design and policy difference. Most commercial renders out there that you can kind of get off the shelf sort of have to be everything to everyone and they're extremely powerful and they're very good but they're kind of designed for sort of a very wide user base whereas Hyperion is has only a single, which is our artists at Disney Animation. And so it's highly custom tailored to our workflows, our artists' needs, our art direction.

So a question I've gotten before is, is Hyperion the best renderer in the world? And my answer to that is, well, I don't know if any renderer is the best renderer in the world, but Hyperion is the best renderer for Disney animation because it's built here for here. So it also gives us a lot of flexibility when the artists want a new feature for a particular show. Well, the people who can deliver that are in-house. So there's no question about, well, how soon will some vendor be able to get to it? Right. So, Carl, you've... Sorry, go ahead.

Sorry about that. You said that you used to work at Pixar. And if, if I, if I'm correct here, Pixar kind of famously has render man, right? That's the main tool that came out of there,

which I think is still being maintained. So you have experience with that as well. Then I would take it. Um, I don't know if there's, you know, anything that you can say, compare or speak to for that,

for people who are familiar with RenderMan. So Hyperion, the origins of Hyperion come from before RenderMan was kind of re-architected to be a Monte Carlo path tracing renderer. So the necessity for Hyperion came from Disney Animation wanting to do stuff that, at the time, RenderMan hadn't quite gotten to yet. Since then, RenderMan has also become a Monte Carlo

path tracing renderer. The internal architecture of the two renders are very different, but the high-level path tracing algorithm is very similar. Yeah, like RenderMan's a really, really great commercial renderer. It's very widely used. But the development of Hyperion and RenderMan are largely separate. The teams do talk to each other. We do share technology wherever it's appropriate. But yeah, I guess they're kind of just... The two renders kind of have different focuses.

Okay. And I'm sorry, I have to ask one other question because being involved in the physics simulation world, I've heard this Monte Carlo thing several times there. And now whenever people are talking about ray tracers, I still don't actually know what Monte Carlo means. Sure. So Monte Carlo integration is basically a way of solving

extremely complex high-dimensional integrals that would otherwise be intractable to solve analytically. So if you think about sort of the rendering problem and you're trying to gather all photons throughout the whole world, you're essentially integrating the contribution of all photons from every light over every surface possible arriving at a point in space,

which is this massive problem where when you expand it out, it ends up having basically like millions of dimensions because it's dependent on the actual scene that you're rendering. So it's totally intractable to solve that analytically. There actually were sort of attempts early in the history of rendering to do that where, you know, the analytical solution essentially took the form of an enormous 2D matrix where it was every single polygon in the world against every single other polygon in the world. And then you needed to find the energy correspondences between every single possible pair of polygons

and when you have tens of billions of polygons like we do today solving this 10 billion by 10 billion matrix rapidly becomes infeasible so the idea behind monte carlo integration is like instead of trying to solve this entire integral analytically what you can do instead is take basically a random sample that represents you know a single you know a solution for sort of a single element of this larger problem and the estimate that it gives you for that single sample is mathematically correct. But by itself, it's also meaningless because it's only one data point out of billions and billions and billions that you need. But if you take enough samples and if these samples are representative and well fit to sort of the underlying structure of the problem, then over time you can build up sort of an estimate of a solution. And this is kind of where the noise comes from

inside of Monte Carlo rendering. When you've only sent sort of a small number of samples, you only have a very sparse sort of subset of the total solution. And everywhere where you don't have sort of enough data to have a picture of the solution, that's essentially where this noise is coming from. So over time, as you send more samples out and you gather more data points on the overall solution, your noise level begins to drop. And then where denoising comes in, David might be able to speak more to this, but where denoising comes in is basically at some point you have enough samples where you can kind of see the overall shape of the solution, but you don't

have the exact solution yet. And then denoising can kind of just step in and fill in what's missing. And the reason for the name Monte Carlo is Monte Carlo is a city well known for its casinos, which are games of chance. And this process is also a game of chance. So it's a random sample of all the possible points that could be rendered. Yeah, technically pseudo-random. So we can actually rerun the render and get the exact same results back

because we control the CD. Oh, okay. That probably is useful. Yeah, it makes debugging a lot easier. It turns out it's actually very important for accelerating the underlying simulation because if you have no idea of what the solution the sort of shape of the solution looks like then random is the best that you can do but if you already sort of know that like oh well this part

of the solution is probably more important or like you know has higher energy or whatever then you can direct more samples in that direction. So your underlying sort of random distribution becomes kind of warped to sort of fit an approximation of the solution that you want, that you might already know. Interesting. So David, then just out of curiosity, like what percentage of the pixels sampled do you need before you can do something meaningful with your denoiser? It depends, I guess, on what your target quality level is. If you're just doing interactive rendering to get a rough idea of is your lighting good or not, you know, how do I need to adjust this light to make it look good, then you might do fine to send like 16 samples to each pixel and then do a quick denoise on it and get a reasonably good result. But for final renders, you'd probably be doing at least 64 samples per pixel.

And in some cases, it could be thousands. Even before running the denoiser? Yeah. Okay. And if you didn't have... Oh, sorry. Sorry, go ahead. If you didn't have the denoiser, approximately how many samples do these...

Like, what's, again, trying to get an ideal for scale? It might be like eight times higher okay interesting want to interrupt the discussion for just a moment to bring you a word from our sponsors clang power tools is the open source visual studio extension on a mission to bring lvm tools like clang plus plus clang tidy and clang format to c++ developers on windows increase your productivity and modernize your C++ code with automatic code transformations powered by Clang Tidy. Find subtle latent bugs by using LLVM Static Analyzer

and CPP Core Guidelines checks. Don't fiddle with command line switches or learn hundreds of compiler flags. Clang Power Tools comes with a powerful user interface to help you manage the complexity directly from the comfort of your IDE. Never miss a potential issue by integrating Clang Power Tools into your workflows

with a configurable PowerShell script for CI-CD automation. Start experimenting on your code today. Check it out at clangpowertools.com. One more question I have about the whole, you know, using this Monte Carlo simulation. What necessitated this change? Like what was Disney Animation using before it built Hyperion? And, you recognize that this was an improvement upon that old renderer? Or did you realize this project you're working on just couldn't be done or would be very difficult to do with a traditional renderer? a few aspects to it. One is the directors wanted a particular look for Big Hero 6 that was somewhat

more realistic and had reflections and soft shadows and so on, a look that was going to be hard to achieve with our previous rendering approach that kind of demanded global illumination. And at the same time, the studio was seeing some other studios that had already transitioned to path tracing and recognized some of the benefits of that for the production process. It's easier to set up and light the scene when you have actual physically plausible simulation of what the light would really do in a setup. The artist can light a scene

more or less like you would a live action scene and put lights where they need to be to provide the indirect lighting and warm up a scene, for example. In the past approach, they would have to place many more lights and be very careful at exactly where they put them because they would be getting primarily just the direct lighting

and not get the automatic sort of plusing that we get with the indirect. Okay. So you're building on these render or rendering on these render farms that you said are massive with many cores. Are you offloading things to GPUs? So Hyperion today is a CPU only, you know, pure C++ code base. We have sort of ongoing research efforts into sort of using GPU compute through CUDA. Right.

But for us, that's sort of an ongoing area of research. Right. I mean, we've had several guests on over the years that talk about CUDA and underlying technologies. That's why I was curious. Yeah. So we're very interested in CUDA right now. We're doing sort of a lot.

But for us, there's nothing that's sort of uh deployed in production yet okay okay and carl uh earlier when we were talking about the visual c++ updates you mentioned uh your religion coroutines what are you hoping to do with coroutines and hyperion okay this is um this is kind of speculative and it's it's a slight of an aside, so I'll try to be quick with it. It's C++. It's fine. So one of the sort of current big unsolved problems in the ray tracing world is... You know how I mentioned earlier that Hyperion is structured in this sort of breadth-first fashion,

whereas most ray tracers are structured in this depth-first fashion. So the way the math is formulated makes it much easier to express in code as a depth-first, sort of one path at a time routine. Most of the research papers on the topic, all the math is just easier to implement depth-first, but breadth-first renderers have a lot of performance advantages for a variety of reasons and a lot of nice characteristics that we want. So one of the big unsolved problems in the rendering space is, is there a way to express these algorithms in a sort of depth-first fashion that maps to the underlying mathematics

really easily while still being able to execute in sort of a breadth-first fashion that gains you all of these nice coherency properties and stuff. And for a long time, I've kind of felt that language-level coroutines is something that can potentially help a lot. Because if you think about the step that you're sorting at in this algorithm, it's basically like you fire a ray, and then that ray hits a thing, and then you need to stop, sort, and batch, and then you can move on to firing the next ray, and then you can stop, sort, and batch.

And if you step back and kind of look at it abstractly, it actually looks very much like a generator model where you have some kind of function that is doing a bunch of stuff, and at the end of the day, outputs a shading point or array that needs to go into a group of other stuff, and then you do a bunch of stuff with it,

and then you want to come back to this generator and then proceed to sort of the next step. And that seems like it's kind of a natural fit for coroutines where, you know, potentially inside of a single coroutine, you could express this entire depth first ray tracing algorithm with yield statements at every time that you're firing a new ray or generating a shading point. And then the coroutine would yield then, you know, return, you know, some piece of information to some kind of underlying scheduler. That scheduler would do a bunch of stuff, and then once it's ready, it would resume the coroutine.

I don't know if this would actually work, but it's something that I'm excited to try. That sounds theoretically possible. Yeah. Do you have a toy or test environment set up in some way that would allow you to easily try re-architecting things without working on this? I'm assuming it's a rather large code base. We haven't talked about that, but it seems highly likely that it is.

Yeah, so in-house we do have sort of our own experimental framework that we test out ideas on. And then outside of that, within the wider industry, there's actually a pretty large number of open-source research rendering frameworks that are very good for trying out things like this. Off the top of my head, two really good ones are TBRT,

primarily by Matt Farr, and then by Wenzel Jakob and his lab in Switzerland. you mentioned also um before since we're talking about software architecture and technology uh that you need to or that you it's pseudo random number generator for your multi-carlo distribution so i i have already been curious like how do you go about testing something like this? Do you test way down at the functional level,

or are you just saying, with these inputs, we expect a frame that looks exactly like this? Testing is somewhat difficult in a big application like this. We do some testing at the unit test level, but the bigger part of our testing is basically just ensuring that we get the same result now as we did in the past, the last time we had sort of a blessed result. So it's just sort of testing the system as a whole.

Right. Do you look for like 100% match? This is exactly the same? We have a tolerance, but it's usually a fine enough tolerance you can't actually perceive the difference. Right. Even with the same seeds, the renders will vary to a very slight amount because we have all this parallel processing going that are accumulating float results into an accumulator. And as you know, with IEEE floats,

the order that you add things in can slightly change the value of the result. Right. You're also talking about an enormous amount of data, right? Because I almost said, is the JPEG the same? But there's no way you're outputting JPEGs with lossy compression. I refuse to accept that. No, we use OpenEXR, which is an open source tool from ILM for our file format. Okay. I assume this supports...

Oh, sorry. An EXR file is basically a 16 or 32-bit sort of lossless, high dynamic range format, which I guess is very analogous to, like if you're familiar with photography, with like DNG files or RAW files. Right. So each frame is huge. It's a floating point format, so it represents high dynamic range data very well. Wow. Yeah, that sounds like a lot of data moving around through your system.

Yeah, and actually, just the geometry, the renderer may have 50 gigabytes of just geometry loaded while it's rendering. So data movement is definitely a big issue. Wow. That sounds like something else that'll be important when you try to do GPU, because that's a lot of data to move out for rendering also. Yeah. Yeah.

It's definitely a good thing for sort of our industry now that uh many of the gpu vendors are starting to put out cards that at the very high end have like 48 gigs or 96 gigs of onboard memory yeah i mean you pay for it but just the moment that makes me feel old i was just remembering the other day that my first pc compatible had 512k of ram before i rode my bicycle down to radio shack and bought that 128k upgrade to get it maxed out yeah my first computer had four kilobytes until i bought more chips and got it up to eight yeah no i didn't have anything with quite that little RAM at any point. What was that? That was an Ohio Scientific Superboard 2 6502 processor. Okay.

So back in the era when it was largely home-built kind of things? The board came already built, but it didn't have a case. So I built a case out of wood. That's the extent it was home-built. But it was great because all the chips were socketed. And so I could actually pull out the graphics chip and extend the wires out to a solderless breadboard and design my own graphics hardware for it. And I did some of that too. It's like the old cars where you can get in and really tune them and do things to them that you can't anymore. I'm pretty sure you're the only person I've ever heard say that you wired up your own

home built graphics card off of your first computer. sure you're the only person i've ever heard say that you wired up your own home-built graphics card off of your first computer it was great the other thing that was good about that computer is that actually came with full schematics so it was almost designed to be able to tinker with it like that nice uh so back to hyperion for a moment um i'm assuming hyperion is an in-house tool not not open source but are there any parts of Disney animation that are open source tools that you work on? Yeah, there's actually several sort of major foundational components to Hyperion and to our overall, to our entire pipeline

that we have open sourced. So there's P-TEX, which is short for per-face texturing, which is sort of a UV-coordinate-free texturing system that was designed in-house and has since been open-sourced and incorporated into a number of other authoring tools and commercial renders and such. I have no idea what any of those words mean. UV-coordinate-free texture what?

So if you're familiar with texturing in games, commonly the way it's done is there's a texture atlas, which is a rectangular texture image, and then each polygon or triangle face has a separate section in that image. And you have to do kind of a puzzle-fitting process of fitting all of your shapes together into that rectangular image,

which is a somewhat difficult process to do well. P-TEX avoids that need. Each face's image is stored in an entirely separate area of the file without having to force them all to coexist in a single rectangle. But again, I assume this is done efficiently because there must have been a reason why other people do shove it all in one rectangle, right? So that they're all accessible in the same chunk of memory or something like this? Or is it just convenience? There's a variety of sort of historical reasons why it's done that way.

Part of it is just it's done that way because it's always been done that way. Okay. And because originally the graphics hardware could only support those rectangular textures. Yeah. Yeah. Right. That sounds like a direct evolution of the old tile-based 2D rendering kind of thing, 2D games,

where you would have one large map that had all of the tiles in it and then you would, you know, draw out the specific chunks. But maybe I'm reading too much into it. Yeah, I think 2D, sort of the normal sort of 2d mapping based approach basically is that okay yeah i'm so sorry i interrupted you you're describing what this

that this open source source tool does yeah no worries um so yeah p-text is basically just a it's a way of texturing that um for our artists it's a lot easier to deal with because they don't have to figure out they don't have to manually have to figure out this mapping between like the mesh and this 2d rectangle instead they can kind of just paint directly onto this 3d mesh and not worry about any of this underlying mapping stuff so that's one of our libraries another one is called PartIO, which I think it stands for ParticleIO. So it's sort of a general purpose utility for wrangling 3D particle data. There's a variety of different file formats

and underlying data representations and whatnot. And PartIO kind of just sits on top of all of them and presents a single unified interface so that you can read in and write out particle data from any source to any other source. There's another library called se-expr, which is a custom expression language that's implemented both as a C++ interpreter and as a, sorry, as an interpreter implemented in C++

and as a sort of a JIT built on top of LLVM. So it's this sort of in-house language that is used mostly by the artists for writing short expressions that can be part of shading operations. So for example, if they wanted to sample a texture map and then modulate the result by some noise function and then put that into some shading channel, one way that you could do that inside of many other packages is you would have this big node graph and each of these operations would be a node that you drag a string to another node. But Se se expert kind of allows you to express all of that as just

a single line expression of you know code um so so yeah that one's also open source it's also you know incorporated into a number of other sort of third-party packages and commercial solutions and then uh in-house, Disney Animation has a whole bunch of other open-source projects that are really interesting. There's one called Monkey, which is essentially a reimplementation of the software update infrastructure in macOS, so it allows us to have a studio-wide deployment of Mac workstations.

Most of the production is on Linux, but we do have a lot of Macs in-house as well. And Monkey allows for sort of deploying our own software packages and pushing out updates to all of these machines. And then aside from that, Disney Animation also has two sort of large production data sets that we've open-sourced. So it's real production data from the film Moana, which we've kind of packaged together into a format that works outside of the studio and we've released to the research community. What do people use that data set for? So for a long time one of the big sort of complaints in the rendering field was that

all of these academic research labs were developing really interesting new techniques and really interesting new algorithms, but they can never really test if they would scale from sort of toy examples to production scale data sets. Okay. Because production data scale data sets require a production studio to put together and to help sort of alleviate that problem and help out all the various research groups out there. I think it was two years ago maybe,

Disney decided to release sort of a set of real assets and a real scene from Moana. That's cool. And you can find more information about all of this on DisneyAnimation.com. We have an open source page there. Yes. Awesome.

Awesome. Okay. I feel like I was kind of building this mental picture that part of your job is to insulate the artist from some of the technical things. And then you're like, oh, and by the way, we have a custom programming language that's justulate the artist from some of the technical things and then you're like oh and by the way we have a custom programming language that's just for the artist and so it kind of blew up my mental model just a little bit there um well if you think about it like a lot of um across sort of all the artists that we have there's definitely sort of a range

where you have some artists that are very very very, very, you know, non-technical. They work, you know, they're focused exclusively on sort of like the actual art side. And then on the other end of the spectrum, we have some artists that are actually extremely technical and many of them could probably be very strong developers in their own right. So, you know, for example, effects artists are often building up these massive complicated simulations that depend on an enormous number of moving data pieces. And these are basically wired up as these enormous complex graphs inside of this commercial package called Houdini. And these graphs are essentially a visual programming language. And because these graphs are basically just specifying data flows and they're specifying like logical orders of operations and whatever.

And so sometimes when these graphs get enormous, it's actually easier to just express them as code instead. So in Houdini, you can, you know, freely flip between this graph representation and sort of an expression language sort of thing called VEX. And so for our shading artists, we provide something very similar where, you know, instead of having like where they can express these extremely complex shading graphs as essentially code instead. Okay. And there was a speaker from CBCon last year, I think,

who works on Houdini, right? Or was that two years ago, maybe? I know I've heard of Houdini before. I think it would have been two years ago if it's who we're thinking of. Yeah. All right. Well, it's been great having you both on the show today.

Is there anything else you wanted to mention before we let you go? Oh, I know. We are going to be hiring soon. If you go to DisneyAnimation.com on the careers page, we don't have the actual job recs up yet, but there's a kind of generic one that summarizes the types of

people that we're looking for and lets you express interest. So watch that space. Alright. Thank you both. Thank you guys. It was great. Thanks for coming on. Thanks so much for listening in as we chat about c++ we'd love to hear what you think of the podcast please let us know if we're discussing the stuff you're interested in or if you have a suggestion for a topic we'd love to

hear about that too you can email all your thoughts to feedback at cppcast.com we'd also appreciate if you can like cppcast on facebook Facebook and follow CppCast on Twitter. You can also follow me at Rob W. Irving and Jason at Lefticus on Twitter. We'd also like to thank all our patrons who help support the show through Patreon. If you'd like to support us on Patreon, you can do so at patreon.com slash cppcast. And of course, you can find all that info and the show notes on the podcast website at cppcast.com. Theme music for this episode was provided by podcastthemes.com.