The Offset Podcast - The Offset Podcast EP004: AI & ML IN Post

Episode Date: February 15, 2024

****** A gigantic thanks to our good friends & new sponsor Flanders Scientific! Visit www.flandersscientific.com to learn more about all their professional monitoring solutions ******  Unless you've been living under a rock, you've likely heard the countless ongoing discussions about 'Artificial intelligence' and machine learning.  In this episode of The Offset Prodcast, Robbie & Joey share their thoughts on how AI/ML is currently affecting the postproduction industry and the possible future of AI/ML in post - both good and bad.

Hi, in this episode, we're going to talk about AI and machine learning and why the colorist is not dead yet. Stay tuned. This episode is sponsored by our friends Flanders Scientific, who are leaders in color accurate display solutions for professional video. Whether you're a colorist, editor, DIT, or broadcast engineer, Flanders Scientific has professional display solutions to meet your needs. Learn more at flanderscientific.com. Hi, I'm Joey Deanna. And I'm Robbie Carmen. And guys, today on this episode, we are going to be talking about something that unless

you've been living under Iraq, you have been bombarded by, you know, traditional news, the internet, friends and family. And that is these buzzwords of artificial intelligence and machine learning, AI or ML for short. Now, just to be clear about something, Joey and I have opinions about AI and ML. But we are neither, neither one of us are experts in AI or ML. We are not coding the platforms to do this. We don't know everything there is to know about them. There are doctors, you know, people who have doctorates and, uh, via, you know,

PhDs or whatever out there who are doing this stuff. But we thought, because, you know, talking to our friends, our clients, our peers, there is so much buzz about AI and ML, um, in our industry in post-production, in color and finishing. Uh, you know, it runs the gamut from editing to, you know, music creation to whatever, we thought it'd be good just to kind of talk about this, share what we think are some of the fears that people have about ML and AI and why some of them probably are unfounded, but some of them might have a little legs to them. And then kind of where

we see things going for the future of artists and finishers and colorists like ourselves. So, Joey, I think the first place that we should start is kind of the everyday layman's understanding of what is AI, what is machine learning? What do you think about those two phrases? I have, as you may or may not know, very, very, very strong opinions on the pedantic terminology related to this emerging technology.

In my mind, we are completely talking about a technology called machine learning. None of this is AI. None of this is intelligent. The computer does not have. have any intelligence. The software does not have any intelligence. It is still algorithmically driven by code written by humans. It is not thinking or creating. It is deriving based on a series of inputs. And it does not become artificial intelligence until it can rise up and try to kill me

and I have to fight it. Then we can call it artificial intelligence. Can I make one note? But right now, we are talking about machine learning. Can I make one note though that's concerning me? As I've noticed, notice there in your setup, you have a Cyberdyne Systems t-shirt on, and you have what appears to be maybe an arm from the Terminator. So I have a question for you. Based on what you just said about, you know, where there's no such thing really as AI, we're all machine learning. Do you think AI will be a thing eventually? I hope not.

I absolutely hope not. But if it does, I'll be ready. So I think it's a great distinction that you made I think a lot of people use artificial intelligence because it sounds a whole lot cooler than machine learning. Machine learning sounds like a certificate that you'd get at community college to like, you know, operate a forklift or something, right?

But like the Terminator also, artificial intelligence sounds a lot scarier than it is. Correct. Because people are terrified of losing their jobs of having the evil robots take over all the industry and that's just not what the technology does. has no actual intelligence. Well, I think that's a good...

Don't be scared. I think that's a good place to start because I, you know, just to give people a little peek of our history, you know, both of us started in post-production, uh, in the, you know, mid to late 90s when there was a lot of heavy iron, literally heavy iron machinery going on in, in suites and machine rooms, you know, big linear rooms, lots of tape decks, uh, that kind of stuff. And we've seen a, you know, an evolution. of course in post-production color audio motion graphics i mean the the idea that you know you could set you know

have an application to fly text around by setting keyframes on you know an app used to be mythology back in the day when people were running you know big expensive henries and kairons and all sorts of stuff so the idea i think that i want to address first is this idea of technology evolving and changing right and i think naturally that's scary to people and the thing i think about most actually is two kind of worldwind changes on our industry. Number one, the jump to desktop editorial systems, right? Moving from million-dollar, you know, CMX, Grass Valley, Sony, whatever, you know, linear rooms to people going, oh, no, this is my Mac with the Abbott on it

and later Media 100s, Video Toaster, you know, whatever it may be, you know, Final Cup Pro, those kind of things, and going, I'm going to lose my job as an editor, because I'm working in this million dollar editing suite and now it's on a computer like what? That didn't happen, right? Yeah, people thought Final Cut Pro was going to bankrupt the entire post-production industry. It didn't happen.

It just made it bigger. It just made it bigger, faster, more efficient, more, you know, this was bandied about a lot, but sort of democratized that tool set to more creators being possible there. And I think that, you know, we've seen a lot of transitions like that, that number one. Number two, I think the other transition

that comes to mind is people being all sort of, were to freaked out and scared about was when we made the, you know, transition from tape-based workflows to file-based workflows, right? People were, you know, I mean, yeah, unfortunately places like dubbing houses and places like that who didn't change probably did go out of business, but in general, I think everybody can say, hey, a file-based workflow is a hell of a lot easier than managing, you know, a fleet of tape decks and the expense of those tapes and that kind of stuff. So change like this, in my opinion, uh, is not anything new to our,

industry and I actually get I get a little worked up when I see news reports and this stuff because in my opinion it's kind of like fearmongering to a certain degree right like be afraid you're going to lose your job and I think it's an easy way to get clicks absolutely and I think it's irresponsible of a lot of you know newsmakers to do that because you know that fear mongering never suits anybody except for ratings and getting clicks etc when the reality of it is the majority of us will integrate and evolve this kind of technology into our own workflows without having to be kind of, you know, scared of it. So I think that's one point I want to make. The next point I want to make, and I think I want your opinion about this, because it's a

little confusing to people. Besides the two terms, MLAI, there's also kind of this difference in what MLAI does, right? I think a lot of the news that we see, the hype, the buzz, is about, generative AI, and that is AI that is kind of seemingly aware of what it's doing. As you said, it's not really, it's just using programmatic approaches to it. But, you know, it's creating art, for example, right? Is that real art or is that not real art? It's doing things like, you know, the new Photoshop ability to like,

hey, I need to fill this area, generate something that would look cool back there, right? That generative AI is one thing. What are your feelings about generative AI or machine learning things you know, these tools that create something? Is that a good thing? It's a bad thing. Do you think it has a place in post-production and finishing for us, even if that's right now, you know, right this moment?

Yeah, I mean, it's definitely the most kind of eye-catching, interesting application of this kind of machine learning technology, right? It's the wow factor. It's like a computer made that image that looks photorealistic just based on a text description. That is amazing. But at its core, what it's doing is it's basically taking a gigantic library of learned information, right? It's been fed all kinds of still images, video, corresponding descriptions, words, things like that,

just a gigantic set of existing data, right? And then the parameters are basically going through a bajillion if-then-else statements and kind of figuring out step by step how to make what you asked for in a effective and convincing way. Now, where this breaks down in terms of calling it something like creative or creating is

all it can ever do is derive from what it's been given. So if you feed it a crap ton of good photography, it might come out with something that looks like good photography. You feed it a crap ton of bad photography. it's probably going to come out looking like a little bit of a worse photographer if you try to ask it to make a photo. If you feed it all of one ecosystem of things,

everything that's going to come out of it is from that ecosystem. So it's like anything else, what you get out of it is going to depend on what you get into it. And what's really interesting for me, for our industry, is not the technical part of that, right? It's kind of the legal, creative, moral part of that because, and there's been a lot of talk about this and right now there's not really a

regulatory framework or even a best practices framework for this a lot of these generative AIs are trained on copyrighted original art without the full knowledge and understanding of those artists so essentially you're just asking a computer to pirate something copy it and wiggle it around into a more different set of pixels now that might be the worst case kind of explanation let's also talk about kind of the really good case explanation of this. Companies like Blackmagic and Adobe that are introducing machine learning tools,

obviously have very big legal departments and have very good concern about these legal and moral issues. And they've all basically said we don't use any user-generated content. We don't use any of your content to train our algorithms. And we don't just go out and scrape the internet to train our algorithms. They're much more targeted. And that does two things. one, like I said, if you put in bad stuff, your results will suffer. So they're curating the inputs to their algorithms with very careful things to make it do what they want,

and they're making sure not to kind of steal people's copyright. Now, in that case, the generative AI, in my opinion, has a couple of really good uses. One, like you said, the kind of Photoshop, fill something in, object removal in video applications, for example, it would need to know what does grass look like to remove, something from grass. And that's the kind of thing where it's a, it's not really an artistic decision, right? It's not like the entire creative intent

of this shot is removing this shadow from this grass, right? So the execution of that image operation doesn't really have a drastic amount of creativity in it, and it's a great thing for the robot to handle and save you some time. The other thing that I think is really

interesting is, if you go back to the idea that I said of machine learning, taking this gigantic pile of visual data, this gigantic data set, and then generating something from it. Like I said, it's not making art, but it's making an image derived from its set of inputs. It can be a fantastic tool for inspiration and brainstorming. Try throwing different things at it and seeing what it comes up with. It has a way more diverse source of information than your single life experience or your single life experience as an artist. So using a generative AI tool, like, God, I shouldn't

say that, a generative machine learning tool, even I'm guilty of breaking my own terminology rules sometimes, but using those generative tools for inspiration, finding themes, colors, looks, ideas that you might not have had in brainstorming, that that democratizes the creative process, right? It lets somebody who hasn't traveled the world see things and inspire their work with all of this different media that has been trained into these algorithm. So that, I think, is like, that's a really exciting use case for generative machine learning. Well, there's a lot to unpack there. And I think I agree with almost everything you said. I will say, I think there's some debate to whether or not, how should I say this bluntly, that is,

is there art to coding, right? And I think that there can be. And I was trying to, my kids were asking about, you know, generative AI, because, you know, somebody had, you know, at school or whatever, somebody had mentioned mid-jurney and they were looking into it, how does it work, and that kind of stuff. And it dawned to me- Well, there's also some creative intent to the prompt you give it and how you word it. So it's not completely devoid of art. Exactly. And it seemed to me when I started playing with it in hopes of finding better uses and or explanations for my kids, I kind of was hit upon something. And I've had the same experience using, you know, chat GPT. The other day I asked chat GBT, I said,

you know, write me a song, song lyrics about XYZ, right? And I tried it a few times with different additives, different, you know, changes to the way I phrased the, uh, the phrase the request, that kind of stuff. And it dawned on me that part of the game, if you will, of generative AI is taking what is under the hood. I think you're absolutely right. It's a series of logic. It's a series of program, programmatic steps that scours different places to figure out

what you're talking about, how you're talking about. about what you want it to look like, et cetera. Very complex stuff that I mentioned at the top of the show. PhDs are involved in and it's beyond me, right? But what I realized about the prompt part about it is that we're taking really programming to the truest whizzywig approach that we possibly can, right? We're saying, just say something naturally in human language, the way you would describe it, however you want, and we'll create something like that.

And it's very much a way of, in my thinking, democratizing programming to a certain degree, because instead of typing in all the lines of code, you're just saying it, hey, I want this, right? Yeah, and I think that's actually an excellent, excellent point, because we've seen in the history of computing a lot of different attempts to do that from like basic and logo and things like that where they're much more plain language programming languages. I've always said that, you know, a computer will only do what you tell it to do. And these tools are allowing people to tell computers what to do in a clearer, more concise, more digestible for them to way. And that's a good thing.

But then my next thought after that was, wow, this is very similar. My interaction with Mid Journey or ChatGPT or the Google tools or whatever it may be, the platform may be, the Adobe Tools, was this is very much the same situation that I'm faced with, you're faced with, and probably a lot of our listeners are faced with. when interacting with clients describing what they want to see happen on screen and for you to change, right? In other words, we are often the AI machine learning algorithm and our clients are the person putting in that prompt, right? And we've talked at nauseam about, oh, client communication and this is, you know, how to interpret this and people make jokes about us being post-production psychologists and, you know, translating what our clients say. that dawned on to me when I was starting this experiment with these tools, it's the same thing, man. It's only going to be as good as the way I describe it and what my needs are out of it.

Right. And a part of that is word smithing and, you know, kind of being descriptive. Part of that is learning the way those algorithms want and accept prompts and modifications. Like, you know, you get big result, big result differences out of mid-jurney by, you know, clicking different, not clicking, but in, in sort of enabling different, uh, you know, switches and such to do different levels of quality and, you know, how long it takes and that kind of stuff. And so that was one thing that got me about this was that like, you know, I think a lot of people who are blah on this might not be the most descriptive people in the world, right?

Their experience with it are going to be different than somebody who's very verbose, very detailed, that kind of thing versus somebody who just goes, make me a flower, right? There's a big difference between make me a flower and make me a flower that has subtle shades of pink, reds and blue, with dew dripping on it, with morning sunlight, you know, like whatever you want to throw at it. Shot with an 85 millimeter lens two feet away, you know? Exactly, exactly. So, you know, the people I think who are having bad experiences with generative AI may not be the most descriptive people. But the other thing I thought about with those people who are having troubles sort of gronking what generative AI does is that, I think it takes a while to, in this sense, learn the literal vocabulary of what these tools can do.

Just in the same way that we've talked, you know, we're working on an HDR project right now. And, you know, earlier you said to me, oh, I think you pushed it too far. And we've talked a lot about like the vocabulary of what makes good HDR grades and not. It seems similar here, right? It seems like there's, there's a scale to this, like the vocabulary of how we interact with these tools is in, and is getting better and refined and different. And in fact, the people making the tools are going, oh, well, you know, we need to take into account when there are one word that could mean seven different things,

right, and how they program that vocabulary and all that kind of stuff, right? So I think it's there. For me, though, Joey, I think generative AI and what we do is right now, like all of it, at its infancy, but for a practicality standpoint, it's really hard to do in video work, right? And I think it's awesome for making a still,

like my daughter had her 15th birthday. She loves Taylor Swift. Sure, I'll go on mid-jurney and go, create me, you know, a picture of my daughter on stage with Taylor Swift with 20,000 adoring fans. And I can do that and it looks cool, right? It looks like my daughter was on stage with Taylor Swift, you know, at a concert, right? Again, there's copyright implications there.

That's true. But, you know, that kind of thing relatively easy to create. When I'm going sitting in the street with a client and the sign and says, you know, I'd really love to have a drone shot coming over the Hollywood hills that reveals Los Angeles with a thin layer of smog at dawn. And you know what? I want that to be, I want it to be very filmic. So I want to, you know, shallow depth of field. Like that kind of stuff, way more difficult to create in motion that's going to fit in, right? And I think that's one of the reasons that a lot of us are struggling with these concepts right now because we're not actually seeing it yet totally in a place that is like oh i can make a one-to-one connection on how that's

going to work with my work yeah and you know it's always going to be like that i don't want to be the naysayer that says oh this will never be perfect because you know technology does move in very dramatic steps but at the end of the day this is generative AI is never going to replace a photographer or a cinematographer for original content or an animator or an illustrator, it might help them along as a tool. They might utilize it, but it's never going to have value as just full origination. That's a great point. And that's kind of where I, like, so I'm thinking about as a colorist and what I do every day,

where would I want generative AI to step in? And I can think of just off the top of my head kind of three or four things that I wanted to do that tools exist to do this, but they're not, you know, tell it to do it kind of thing, right? So number one, I think it is the most obvious one is fill and object removal, right? You know,

just be like whether it's a sky replacement, whether it's, hey, remove that lamp from the background and fill it with part of the table that looks natural without having to do all the roto work and all that kind of. I think that's pretty simple. I think two, it's about, to me, it's about lights. I would love to be able to go, hey, AI tool, a machine learning tool, I would like to have a tungsten light, you know, a diffuse tungsten light on the right part of this frame.

And not have, and to me, it would be perfect is not to have it just do it, but then to present, okay, here are a level, here's a level of control that you have over to further refine. I've done the heavy lift for you to make it happen. But now you're the artist. I'm just a stupid computer program. Now you refine. That to me is the difference between a usable professional tool

and a toy, right? The straight text in, image out is a toy or a professional tool for finding inspiration that you then will throw out and then do something on your own. Having the ability to go in and change it afterwards,

which is a very tough challenge for the developers, right? Because you don't have a lot of flexibility as to what these systems output. So having the ability to go in and customize after the fact and refine without losing the bulk of the work that has been done, that is the absolute key killer aspect of this that we need for any kind of machine learning tools if they're going to be used in professional post-production. Well, I had a great talk a couple weeks ago with a friend of mine who is, I mean, deep, deep, deep into the world of Unreal Engine, right? and his job has been creating photorealistic imagery that will go on LED volume walls, right?

You know, so if, you know, whatever, you're filming a show and somebody's supposed to be on a ship at sea, he'll do all that stuff in Unreal Engine, and they'll put it on the LED volume wall, they'll film in front of it, and nobody knows that it wasn't practical, right? His take on this, I thought was pretty interesting that he thought generative AI in, you know, again,

he's using one platform Unreal Engine, which is popular for this kind of thing. But his take on it was partly inspiration, like you said, but partly also just to get to the result quicker, right? And, you know, is making the point, if I can design, you know, I can design a mountain landscape from scratch, but it would take me a while to do that. If I can integrate generative AI into it and say make the mountain landscape, but then go back in with the rest of my tool set, finesse it, finagle it, whatever. That's what he's looking for. And I thought that was a wise thing, as you said, because that makes it not a toy, right?

Being able to go... Yeah, let it handle the heavy lifting of stuff that's in the background that might not be the direct focus, but still needs to be there. Absolutely. So I think generative AI, we're going to see that constantly developing. I think the hybrid thing, like I just said,

with Unreal and LED walls, that's there. And I think it will get better. Right now, I think in practice. practical terms for what we do in post-production and production, I think what gets dismissed a lot is, and you say this to me all the time as I'm clicking like 39 buttons, right? You go, why wouldn't you just write a script for that, right? But you hit, when you say stuff like that to me, you hit on a very valid thing about what we do as professionals day and day out, and that is repetition, right?

I can't tell you how many times a day I, click on the same button or how many times a day I make the same set of exports or files for a different project, right? So I think one of the places, and I'm curious to hear your thoughts on this because you're Mr. Efficiency, you know, is where machine learning AI can help us with repetitive tasks because, you know, I just envision a situation, the file thing comes to mind when we're finishing a show and we have 74 deliverables to make, right? I would love a situation where one day DaVinci Resolve goes, just prompts me and says, hey Rob, I've noticed that you're on the deliver page. Last time you made all these exports.

Do you want to do that same thing again? Without me having to go and click and make all the choices or set up presets or whatever, it just kind of intuitively knows where I am in the process based on what I've done that proceeds where I am at now, and then intelligently prompts me to do something. Just kind of like, you know, my Amazon Alexa.

So we'll go, hey, I noticed that you're out of shaving cream. Would you like me to order some more? You know, like that kind of thing, but taking up the next level. Yeah. And to me, that sort of efficiency booster is, in my opinion, the absolute best use case for machine learning technology. And not only post-production, but in any professional industry. Because repetitive tasks are wasted time and wasted time is wasted money. Right.

when Ford makes 50,000 Mustangs a year, they have a robot do the same welds on every chassis every time and put the same bolt in the same places. And a lot of work goes into making those efficiencies so where things can be repeated, where you don't need human input, it's all automatic.

And like you said, any time it's like, oh, I'm doing this a thousand times, just write a script. That's always my answer. And to a fault, because I'll, if I find myself doing anything that's repetitive, My general rule of thumb is like, if I feel like I'm working too hard for this, I probably am and something should be optimizing it. I've found myself go down ridiculous rabbit holes where I will spend like eight hours trying to write an incredibly elaborate script to automate a simple task to save me an hour of time. Now, part of that is my own kind of emotional attachment to not doing repetitive boring tasks and trying to find an interesting.

solution to that problem at the same time. I enjoy that. But part of it is also really saving time. And the value judgment is, well, is this going to be a set of repetitive tasks that I also have to do again in the same context? If it is and I have to do it often, then yeah, it's worth the eight hours to write the script.

If it's a one-off, probably not. And those little cases, those one-offs are where this kind of machine learning stuff could really, really, hugely be useful. Like you said, democratize the programming. If I could type into a prompt in resolve, hey, take five seconds off of every gap in this timeline. Those are pretty simple instructions, right? It could parse what a gap is,

find all the gaps. It could parse what five seconds is. Take those gaps out, right? That would be a manual process that I could easily automate just with a prompt. And that kind of potential, like you said, also having it kind of keep an eye on what you're doing and look for patterns and find things, I envision it taking that even to the next level. Kind of like, God, what Microsoft tried to do so many years ago with the stupid paperclip in Microsoft Office, it looks like you're writing a resume. Would you like a template?

No. But it looks like you're rendering a PBS show. Would you like us to pull up the specs that you used for the last one? Absolutely. And I mean, already we see. Or even more so, here's another one. It looks like you probably wanted to render this as ProRes HQ, not regular pro res.

Right. Are you sure because you did everything else in this project as HQ and you might have just and that'll save me a re-render if I know if it spots me screwing up. Dude, I cannot tell you

how many times that little bit of machine learning programming in, for example, in Gmail has saved my butt not just grammatically in what I've said,

but also saved me so much time. Like I'll type something and be like it, you know, know, tab to fill in the rest of the sentence, right? Like that auto fill kind of thing. And it seems stupid, right? Auto fill. Like, that's not the same thing as it.

It is absolutely the same thing as, like, it's figured out in a predictive way what you're doing, what you want to say, how you're going to say it based on your words. You know what's funny? I was writing an email to someone that was, let's just say, quite terse. Okay. And, uh, Microsoft Outlook and its new modern machine learning spell check literally came up and said, are you sure you want to use that tone

and gave me suggestions for not being so like mean in my email? Now, in that email, I decided to just be mean and ignore it, but I thought that was really, really interesting. Yeah. It was right. I was right in a mean email. And I think that the approach is so far

to what we do, rightly so, have largely focused on the creative aspects of what we do, right? So there's plenty of shows. tools out there now that are doing MLAI, you know, shot matching, for example, right? Figuring out the tonality of one shot or a reference

and matching it to another. It's cool. It saves a lot of time because a lot of the time what we do is we're trying to match stuff. There's a lot of tools that will do things based on identification, right? So even in Resolve, like the faces feature, right? Hey, just create some bins for me automatically based on

who these people are, right? Super useful. I think those kind of tools to getting... Speech to text. that works. Totally. You know, we've had speech to text for 20 years. It's never worked until machine learning technology. And I think that the, the programmers are right to focus on those workflow enhancements

that make editorial color, all that kind of stuff better. But I think there's still a really large workflow gap, right? Honestly, like that exists for those repetitive tasks that are the next level up from things you can do with, you know, droplets or drop folder. or even basic scripts, right? Like, the one that, you know, you and I talk about all the time, especially when we have difficult conforms, is I would like, you know, a machine learning tool that I can just say,

okay, man, here's a folder of media, here's the reference I got from the client, go figure it out. Do it. Go figure it out. Piece it all together. And then tell me what you couldn't figure out. Right. And it doesn't necessarily have to be based on what we've always thought.

Time code, real number, etc. It goes, no, no, no, this shot didn't have any time code. It didn't have any real number. I just looked at the shot. Right, I looked at the shot and I figured out how to match those two up. Have it eye-matched sizing to a reference. Totally. We're already seeing this a little bit with there's some tools to say, hey, what is that font?

Yeah. Which is really useful. Yeah. You know, so seeing those kind of workflow tools evolve, I think, you know, that brings me back to kind of the original topic of this episode is the color is dead. Is the machine learning robot going to take away our jobs? And are the studio going to say, you know what? Robot T-800 is going to grade this next series? No, absolutely not. But it's going to make our jobs faster and it's going to let us spend more time iterating the creative. That's my key here is like, okay, if we can have the machine

learning tool that fixes our conform or helps us with our conform and builds masks for us and helps us make deliverables, like you said, right? I don't want to do an eight-hour project in four hours now. Right. I want to do the same eight-hour project with four hours less administrative nonsense and four more hours focusing on

iterating the creative. Because the more passes you do, we always work in passes. And the more passes you do, the better it gets until you run out of time. So this just gives us more time to be creative. And that is what I'm excited about. And to

flash back, to what we I said at the beginning of this episode about how people you know there was lots of periods of time where people were afraid of that technology change right and what were they really afraid of they were not it wasn't yeah sure at some levels I'm going to lose my job and I'm going to be out of a job right really what a lot of that was about was I am stable and comfortable with my knowledge base and where I am right now and I'm scared about learning new stuff and how to integrate that into what I do and so this just like anything else I think is the people who are going to move most successful are the people who are

interdisciplinary about this kind of stuff, learning about generative and repetitive AI tools and machine learning and how to integrate that stuff best into their workflows. I can see a time where, you know, the best, you know, colorists and graders and facilities in the world, they're the best because they also have some of the best generative tools or they have some of the best repetitive tools or whatever, right? just like we saw a huge leap when, you know, in 3D animation with, you know, tools like Maya and those render, you know, render man engines and that kind of stuff, that technology helped that industry do things that were better. And I think the same thing is true with us. I mean, I would just love that, you know, the other one I think about all the time, going back to some of the machine learning stuff that I think is just there.

I would love to be able to say set skin tone, you know, set the look and set skin tone on somebody. and go, hey, look at the rest of this group. Keep it going. Keep it going. Like, make sure. I've made the creative decision. Yeah. Keep it going for me.

I'll come back and check in on you when you're done. Screw Rob screwing up ripple for the hundredth time this week. And just let the machine learning figure out how to ripple that change to wherever it needs to go. And better yet, I think, you know, I think prompts me are a big part of this, right? You know, if I make a decision without the help of the system, right? I go, well, I'm going to make this person a little more olive complex or whatever. You know, the system going, hey, look,

I've looked at all these shots and I think that you're making the wrong choice somehow, right? You know, like, you know, or whatever, some sort of prompt by saying, are you sure you want to change this? Because this seems correct. And that goes back to the core concept of what this is, which is a pile of information and a prompt or some user input put together in a very complex web

of programmatic if statements. And where that can be super useful like we've been talking about in a post-production software is if part of that information is the functionality of the software. That's what we haven't seen yet. We've seen individual tools where it's like, okay, we're going to make a depth map.

That's what it does. We're going to make a mat. That's what it does. But there's no tool that knows all of the different things that resolve can do and then also knows how to interpret English. So you could say, put in a prompt, hey, add every timeline to the render queue for me. It knows what the render queue is.

It knows what timeline is. It knows what's the word add means. In this cloud of inputs, it has all it needs to save you time. That's the kind of workflow tool that I think could be really, really cool. And it's never going to do everything. But just to have it, I think would be awesome.

Yeah. And I think the last thing that I'll wrap my last thought on this is I think you brought up something that's above and beyond our pay grade and knowledge. but I think is really developing fast and interesting to see how smart people figure this out is kind of the ethical and legal concerns about some of the stuff. Because one of the things that I'm always thinking about when I think about this subject is, as you said several times, the tools are only as good as what they're being fed, right?

You know, we did a project a number of years ago where it was colorizing World War II black and white footage. And at the time, you know, it was a really heavy lift with, machine learning, you know, giving as many World War II images as possible to generate mats. And it was only as good as what we were giving it, right? And we had to, you know, scour for thousands and thousands of images. I think

part of the ethical and legal part of this is that the powers that be need to figure out a balance between privacy, copyright, and also feeding those engines in a way because, you know, if there was... And transparency. So you know what's going into what it is. And if your stuff is being used and you agree to the

context. And a way of pulling it out if necessary, right? Like if you accidentally, you know, hit scan my timeline for information and it's a, you know, music video that, you know, is not released yet and the artist is going to be pissed by that. Like, you know, yeah, don't share it. I think there's that balance is yet to be found. But I'm curious as the stuff works through these ethical and legal concerns, how that kind of stuff works out. Because I think the data, the data is unbelievably massive in there. And, you know, right now we're going with a slightly small, you know, hopefully, you know, sort of controlled approach to getting some of that stuff.

But the floodgates will be open when everybody's stuff is just there. But I just want to do that in a smart way. Yeah. And in the same vein of the ethical application, I think it's really, really, really, really important to make a distinguishing comparison between what a general. fix of an image or a generative improvement of an image is in context of reality, specifically with regard to documentaries and historical preservation and stuff like that. I won't say problems, but big questions and implications to think about with a lot of these

machine learning based, for example, rescaling tools. If you're doing a historical documentary and you are scaling up a piece of film, using a generative AI tool, you are adding in information from the modern world that was not there. You are asking a computer to take piles of modern images and make something approximate to what this was, but you have now, in some way, devalued the historical accuracy of that media. And it's the same thing with colorization. It's the same thing with machine learning base, like sharping and noise reduction. Those details that are being put in.

are fabricated. They aren't real. They were never captured in the original captioning. I'm not saying there's no use for that and it should never be used. I'm saying documentarians should be very aware

of what the technology is doing and the implication to the historical accuracy of archived images. And it gets worse from there, of course, and not, we don't have time for this today,

but like, I mean, already there is the ethical conundrum of deep fakes and, you know, how that kind of stuff works and kind of creating news and, you know, putting more... But those are so easy to pick out,

and it's so easy to write a program to pick them out. What gets me is, like, if I'm doing a documentary and I've got some old film, and we scale it up using the best machine learning-based scaling, wow, you can see the detail in his face. Well, that's detail that was never captured. We've just made up detail in that guy's face. Is that the right thing to do in a documentary?

Only the documentarian can answer that question, but they should be armed with the knowledge to ask that question. And I think in a lot of cases, people might be excited by this new technology, think it does magical things, and not realize what's going into it and the kind of ethical honesty implications of that. So, the colorist is dead. Long live the coloris is what I'm going to say. I think that, you know, it's always been the case that those who adapt and learn are going to be at the top of the heap versus those who, you know, grumble about it and don't want anything to do with it and refuse to learn. So I would urge everybody, you know, again, Joey and I are not experts. We don't have PhDs in machine learning and AI. This is just our opinions as users.

But I think I look at it as an exciting time to be in, certainly interesting for a lot of legal, ethical, whatever, you know, reasons that, again, smarter people will hopefully apply and figure out. But I think us as users, whether it's repetitive, you know, repetitive fixes, whether it's, you know, using generative stuff in a control. old way, whether that's inspiration or fixing problems. I think we're just at the beginning there. And I think that it pays for all of us to kind of, you know, do our research, think about this and try it out. So plenty to think about with all this stuff, Joey. I think it's been a good talk.

And for the Offset podcast, I'm Robbie Carmen. And I'm Joey Deanna. Thanks for listening.