The Standup with ThePrimeagen - FFMPEG takes a Big Sleep

Episode Date: November 9, 2025

https://twitch.tv/ThePrimeagen - I Stream on Twitch https://twitter.com/terminaldotshop - Want to order coffee over SSH? ssh terminal.shop Become Backend Dev: https://boot.dev/prime (plus i make courses for them) This is also the best way to support me is to support yourself becoming a better backend engineer. Great News? Want me to research and create video????: https://www.reddit.com/r/ThePrimeagen Kinesis Advantage 360: https://bit.ly/Prime-Kinesis 00:00:00 - Intro 00:01:23 - the issue 00:04:01 - the takes 00:09:16 - the bug 00:10:22 - AI Disclosure 00:11:42 - Prime weighs in 00:13:28 - Disclosure credit 00:14:40 - Big Sleep 00:15:24 - AI Bugs Finders vs Static Analysis 00:17:40 - Do they owe them anything at all 00:19:40 - Is this optional or worthwhile 00:23:33 - Bugs in general 00:26:08 - Triage timespend 00:27:01 - Outro

Microsoft is a corporation that turns CEO claims 30% of new code as AI into so many more abilities, dude. It's crazy. Hello, chat. How are we doing? Good afternoon. That's a great intro, Ed. I really appreciated that intro. Yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, sorry.

All right, all right, welcome. Okay, guys, don't interrupt you. You can't laugh about it. I'm just asking if you guys are ready, okay? Welcome to the stand of today. We are talking about FFMPEG and security. We brought on with us resident security expert and Timu Casey low-level learning. That's crazy.

Hello. Hello, chat. How we doing? What an insult. I'm just taking your words. I set in Prime's chat. I set the stage. It's okay.

I'm not insulted. It's true. I am T-Mu-Casey. To be in the same category? T-Mu-Casey. Dude, I'm Casey for the low-low price of $499 made out of plastic manufacturer. Beijing. That's pretty good, dude. That's good. I don't even get that. Okay, like,

I got none of it. Or we always have with us Teage, we always have with us trash. Everybody say hi. All right, so Casey, you go to, Casey. Sorry, wrong. Wrong person. This is going well. We're crushing it, bro. Oh, man.

Just walk this dog to the park. Tell us how, what's going on. I don't know. Let's do it. So, yeah. Hey, my name Kimu Casey, also known as the low level. So yeah, I'm a security guy, right? And you've probably seen maybe a few of my videos about the world of like AI, CB.

Can you zoom in a little bit? Because you're pixelated, and it's like impossible to read that tweet. Trust me, I'll get there. I'll get there. He's streamed before, bro. Bro, I know how to computer, technically. Anyways, FFMPEG is this piece of software you're probably all very familiar with. It has kind of two components.

It has the FFMPEG software stack, open source volunteer project. It also has the FFMPEG Twitter, which is. kind of a whole other animal. It's just two parts of the project. In and of itself, right? And so FFMPEG's Twitter has got a little bit of heat recently. They've kind of stirred the pot for Google. So Google has a couple of projects to find vulnerabilities in software.

One of them is OSS-fuzz. Basically, they fuzz or inject bad data into open-source projects, wait for bugs to fall out and report them. And their new push is to do LOM-based security, right? So they use a giant L-L-LM to read software and then use the reading of the software to maybe find like use after freeze kind of more complex bugs that fuzzing has a harder time finding um and so google did report a vulnerability it found a vulnerability ffeg peg and it was submitted um via their AI system and uh this kind of pissed off the ffm peg twitter guy again i don't know if this guy's

actually a maintainer or if he like runs just a twitter or if he's a bot i have no idea um but basically his argument is that google submitted a vulnerability but they submitted it with a 90 day disclosure period. It basically says, hey, this bug is private. We're not going to tell anybody until 90 days have elapsed. After 90 days, if there is no patch, we will make it public, go and do with that what you will. And this kind of started the conversation with FFMPEG. FFMPEG is saying, hey, if Google has the compute to find vulnerabilities, they should also be patching the vulnerabilities, right? It should not be on a volunteer organization to be at the whim of this company that has all this compute and kind of be held at gunpoint by Google, right?

And so it kind of started the whole conversation on Twitter about not only the security disclosure timeline, but also like, is it right that multi-billion dollar corporations use FFMPEG, Netflix, looking at you prime, you know, probably a prime candidate? So I'm kind of curious what you guys think about the situation, right? Is it wrong that Google submitted a bug report to an open source project using large amounts of compute? Is it Google's responsibility? to find the vulnerability and fix it. What do you guys think? And I think we may have lost prime,

but we'll keep going here. I want to hear Trash's take. Yeah. You want to hear my take. Yeah, I do. I want to hear what you've got to say, Trash. I personally think it's fair, especially something as popular as FFMPEG, right?

If anyone's consuming it. I don't, the 90-day thing's kind of interesting because, like, they call it as a volunteer thing, like they shouldn't really be subjected to some timeline. But I guess there is some responsibility for having a project as popular as F of MPEC, right? So, like, I see it from that angle, but, you know, like, how are, how do they hold them accountable to that?

Like, I mean, sure, they go public with the bug, but does that mean this stuff like that's about it. It's not really like, pay us by 90 days, we're going to kill your family. No, it's literally just like 90 days we'll go by and then we will publish the bug. Brut down. What happened? I don't know. Did you push on a Friday?

Never. How are we going to figure this out? How are we going to figure this out? That is your prods down? Do you guys see the wheel? You have them to pick. That makes sense. I'll spend a couple hours for factory.

Oh, beat the plugins. Don't guess where your issues are. You can see exactly where they are happening with Century. Get all the context you need to debug any problem. Because code breaks, so fix it faster with Century. So this kind of started the conversation on Twitter about why we even have these disclosure timelines. And the rationale behind that is this is a bug in software. Whether or not we discovered the bug, the bug exists.

Right. So that means that right now in reality, like someone is born. to the vulnerability this exposes. So we have an option here. We can disclose it publicly now and let attackers and defenders know about it, or we wait, try to mitigate it, and then give a disclosure after it's been mitigated. But the problem is, if you release it before the mitigation, you don't have a patch, and now everyone knows a bug exists, right?

So you kind of heighten the likelihood that it gets attacked, right? Right. Like, I don't get that threat of, like, making it public, because it just does more harm than anything. like why don't they just help fix it, right? Well, sure, and that's kind of the question is like why, like this bug is not, not that it's simple, but it's not that complicated. It's a use after free, right? Which basically means pointer gets freed, still has a pointer variable in it, but you can use that pointer somewhere else. A simple, like, pointer equals null after the free would literally fix this.

So the question kind of becomes, why did Google not fix this? You know, why did Google who has resources to find the bug, not just fix a bug at the same time? Um, my thought here is that just like literally they found it in an automated fashion and just submitted the ticket and like no one was like the process of finding the bug did not also include mitigation because I don't think they had the scale of people to do that. I'm not sure if big sleep, um, can find and mitigate bugs if that makes sense. But what are your thoughts to teach? Yeah, I mean, it's that there's definitely like a given take here of it's nice to get bug reports that are. reproducible and can tell you exactly what happens and how to do this and what the path is to fix. That's like that is a valuable thing that Google's providing. So that's nice.

But it does feel like really crummy to be on the receiving end of them where it's like, one is not super obvious. I don't know in this particular case, but at least generally, like, is this really even exploitable? Is this really like a thing that actually could happen in practice. Is this like, you know, does someone have to do something insane? Like, oh, this only happens when someone tries to cut out minute, you know, seven in a 49 hour video or something like that, right? Like, um, and so especially I can understand the difficulty of being on the maintainer side and you're getting effectively like an automated LLM bug report from Google and Google's just like, well, you guys should just fix this, bro. Like, bro, just fix it. Just do with it. And you're like,

Regers just deal with it and we're going to... You don't have billions of dollars of recruiting spend? Like we do? Whoa. What do you mean? That's not possible. And so that part and the... Especially like from the maintainer side, they probably see this like upward slope of how many times they're going to get issues and disclosures like this where they're like, how could we possibly keep up with potentially the volume of how many things Google could be like, send.

or submitting to us all the time. And that's like, that's just Google. What if, like, Amazon wants to do the same thing and Netflix wants to do the same thing. And, and now you're like, I have, like, 25 of the, you know, Fortune 500 sending me bug reports every day of things that we need to fix and then no, like, follow-up situation or, like, no maintenance fee or nothing. It's kind of just, like, just fix it. I mean, the software is, like, I mean, I don't know what FFMPEG's license is, but I'm assuming

it's provided without warranty. So it's like... It's in the license. Yeah, yeah, right. That's a good question. That's a good point you've brought up. And that's actually what FFMPEG highlights, I think, in this. So to kind of bring up trash this point and your point about like, how exploitable is this?

How much does this matter, right? Yeah. So the vulnerability is actually in a codec written by a hobbyist from 1995. Or not that the code is from 1995, but it supports a codec that has only been used in 1995. It is a proprietary LucasArts codec. used in a video game, Rebel Assault 2, for the first 20 to 10 frames, 10 to 20 frames, and this is basically the only use of this codec

that we're aware of, and Google submitted an AI report to it, right? So it's kind of a two-sided sort. It's like, okay, they're spending all of this compute spend and basically bombarding open source with these issues, comma, but also no one is vulnerable to this vulnerability. So, like, why are we talking about it so much? Why has it gotten so much attention? And I think it highlights to what you're talking about, the increasing slope of CVEs.

I think AI, well, it's not good right now. At bug hunting, it's going to get better. It's only going to keep getting better. And so for an organization like FFMPEG, if they get a thousand bug reports from Google's Big Sleep, what are they supposed to do? You know, like, it's a good thing naturally that they found the vulnerabilities. But now what? You know what I mean? What are they supposed to do with that?

Yeah, the thing, too, is like, and I get it, you know, the public, like, like, Like at some point they want to make it public. They want to be able to say, we've found these things. They feel like they need to do some disclosure of it, etc. To let people know who are or potentially could be affected by it. But it's also kind of like sending it public makes it so that attackers now target this thing who don't have access to a trillion dollars of GPUs and all of the latest things to find the bug. So like maybe attackers would literally never find this bug except that now Google's like, hey, here's a bunch of unfixed security issues. FMPEG, like in case anyone was wondering, we've got a full platter.

Try and exploit them if you like. Yeah, and so that's the whole argument about disclosure, right? It's like, you are disclosing enough bug exists so that defenders can do the defender thing, but also you are now removing the act of vulnerability research and just giving pox to attackers, right? So, yeah. Not sure there actually is an answer to this question. I just think it spawned a very useful conversation on Twitter. And I would like, you know, people, I think it's, I'm also,

curious to hear people that are not in like security spaces say because obviously I have a very biased you know potentially myopic lens of how I look at this but you know people actually maintain software as their day job it's probably very different so prime what do you think about this man so here's my big problem with the whole situation and really responsible to close disclosure overall is that none of this takes into account who or what they're disclosing upon which I think is I don't know if you guys talked about, but I think that that's just like a huge, it's just a huge problem, right? Because if you have, say, a product that everybody uses and it's a paid-for service in which

company has actual engineers and it's their jobs to maintain, then yeah, I feel like you are more allowed to be like, hey, you need to fix this bug and we are disclosing this. But what does, like, Google gain about disclosing? I always figure disclosing is a mechanism in which you try to pressure someone to either, A, make a fix or B, you want to get the street credit for being able to find this really and unique novel security thing so it helps your resume this is just google's big i right they don't they don't they can just be a chart we found 47 critical bugs and 14 different you know os s projects boom like that they only need high level stuff so i don't understand

even google's like pressuring to begin with the thing that they're pressuring against isn't created by people being paid or full-time employees so it's like all of it just seems just wrong in itself because there's no individual person that's actually getting um like some sort of street credit. It's not like Bob from, you know, the security department found this really cool 1995 Kodak issue. He was playing some Star Wars,

realized something and hacked his mainframe. Like, it's just like not happening. And so that's, I guess that's where my confusion at is, why is Google doing this to these people to begin with? Yeah. So, I mean,

it's a good question. I don't think that the disclosure process is a process meant to give people credit. I think that is a side of thing. of disclosure, right? I think when a researcher discloses, it is supposed to be that they are disclosing to tell the vendor and to tell the public, hey, your stuff is vulnerable, comma, also look at me, I found this bug, right? If you're doing disclosure for clout, that's a separate conversation. Now, to your point, this is not done by a researcher. This is done by an AI, right? So,

you could argue that, like, Google is getting clout, but realistically is... Very human of it. What's that? Give me the credit, bro. It's the most human thing of Google A's... I want it right now. Yeah, exactly. Maybe they said chaos orbs were on the line. You know what I mean? Like, I don't know. And it's like, I got to put this on Twitter right now. I got to go talk about this.

No, I mean, that's the thing. Disclosure is about telling, I don't know if you miss this prime because you may have been out with the internet issues. But disclosure is about telling defenders. Disclosure is about like, hey, whether or not we report on this publicly, the bug exists, right? And we know about it secretly. Attackers may know about it secretly, right? So we have to inform people that this bug exists because you may be, at risk of its exploitation.

Now, that's kind of the problem with this codec, is that, like, literally nobody has this codec enabled. Like, this does not compile by default into FFMPEG. You have to explicitly enable it, and the only place it's known to be used is in a game from 1995. So it's like, to me, it feels like Google's AI is set up to Fuzz Open Source software.

That's what OSS-Fuzz does, and I'm sure that's exactly how Big Sleep operates. And so the AI probably triages all the codecs that exist, and it found a bug in a codec, and it gave it the default disclosure timeline. And I think it's the combination of like all of those factors, like the automation

behind it, the lack of customization to the actual attack surface that piss people off. It's like you spent a billion dollars or whatever on this niche codec found a bug and now it's my problem. You know what I mean? But it does highlight a larger issue

that you highlighted like how does FFMPEG deal with us at scale? That I don't know. So two things. One, there's this whole AI finding bug thing. I know you're saying AI is really great at finding bugs. They have been pretty good at finding bugs, you know, especially if you give them stack traces and you give some sort of hinting to them. But generally, all the security reports I've seen where things are just like, go find a security bug, have all come up as,

you know, predominantly noise. I know Daniel, gosh, I can't, I can't seem to remember his last name from Curl is just like, you don't submit AI generated reports. You are not allowed to do this at all. And he gets just constantly bogged down, a bunch of people are getting bogged down. So I don't know how convenient these things are comparatively to something some coding guy suggesting, which is Coverity. What's your experience with Coverity at finding, say, use after free bugs comparatively or static analysis tools in general compared to these AI things? Yeah, so a static analysis tool will always be concretely better, right? Like, it will always, like, if Coverity says there's use after free, there is use after free. There is no question.

Yeah. The problem is that is basically a one-for-one scaler against human talent, right? Like, you need the engineer equipped that knows how to use covariity, that knows the codebase, to use covariity and to triage if they have to freeze actually there, right, which it likely is. The attempt with LLMs is to take security talent, which does not scale well against software engineering talent, and scale it. Right. But the problem that we run into is kind of what I think whoever said that in chat is talking about is the problem that Sean Heelan talked about in his article where he basically found a zero day with LLMs,

but the problem is that the, let me see if I can find the verbiage here, the signal to noise ratio, which is basically how many are real versus how many are fake, of LLM generated bug reports is 1 to 50. So you may ask an LLM for, find me a use after free vulnerability in the SMB server for Linux, and it will give you 50 reports, all of which detail this huge stack and like this is the bug right here. Only one of them are real, if that. You know?

So that's what I say like LLMs are getting there. They're not good yet. Like an LLM can find a bug in software, right? The problem is just like how do you deal with the scale of that? Because now I have to triage 50 potentially fake vulnerabilities, you know?

So yeah. And so and like Google and some of the other places, they don't have any like contractual things with FFMPEG or anything. That's what seems crazy to me. Like I feel like wouldn't you

if you're using it like an insane amount for that core of a thing? Like I get it that like okay maybe you know like left pad 37 you can't figure out that it's an important like dependency for your company and so you don't. It seems like they don't have people funding like FFMPEG time or something. Like I literally have no idea.

I couldn't find or see anything there where they had like maybe some SLA or whatever with them. Yeah. I am not aware of the contractual or legal nature of this arrangement, but Google does spend a lot of resources on fuzzing open source software. This is a project, Google OSS fuzz. Literally, the Google team spends Google compute in Google Cloud Platform to constantly be pulling, setting up harnesses for, and fuzzing a variety of open source software, which is, like, for example,

FFMPEC is literally a target inside of OSSFuzz. If you go to projects here, you'll see they have all these fuzzers and harnesses set up for all of this software, one of them being FFMBank that I'm getting to. And so they have been spending a lot of money on finding vulnerabilities, right, which is a good thing, naturally. But like, where does that tradeoff happen? Right. Like, when do you now also help mitigate the vulnerabilities? Because if you're acknowledging that, like, you are a big company, you are doing this altruistic thing to help make the world a safer place. But you also acknowledge you have more money than every volunteer organization.

like how do you go about, you know, rectifying that? And the answer, in my opinion, is like, spend some money on mitigations as well. I don't think that's been done so far. And that's the whole argument that FFNPIG is making is like, hey, just give us a, you know, a patch file or something. Can I ask a quick question? Yeah. What do you think the chances are that trash is looking at Pokemon or paying attention right now? That's so, okay.

I was looking at the read-me for OSS Plus, and I was trying to understand how do they, how, how does a project qualify to be fuzzed by this? That I don't know. That I do not know. Ash, great question. See? Doubtors? See, what if they, what if they,

when they start fuzz in my code base and they start sending me stuff? I'm like, whoa, buddy. I don't, I don't maintain this code no more. Do projects opt in to be a part of this Google Sleuth or whatever they call it Google? That's actually a great question. I do not know. I do not know the process there. Because I don't like that because it feels like FFM Pague should be able to say,

hey, we don't want these reports. Like, you shouldn't file CVEs against us. Well, I mean, so I think the argument there is like, why would you opt out? Yeah, I think they want them. It's just there's, there's probably a signal the noise. They wanted them. They wanted them to fix it. They didn't even want, like, they're like, oh, well, just set us a patch.

They don't want the report. They want the result. Well, sure. I think in this particular case, I'm not sure if the complaint was generally speaking that they want like Google engineers just sending a bunch of fixes for everything. but like in this particular case it felt like okay the fix is relatively simple you know Ed you gave an example of how one way

that you could fix it like quite easily like don't send us a report just send us like a fix for this yeah and then like because that makes sense but there's going to be used after freeze or other like actual you know slightly more complicated vulnerabilities that they could find that like you would still like I think to get those free so like for NeoVim

I know we've had I'm pretty sure Coverity scanning for a long time and like used that. And for large open source projects, I'm pretty sure it's free as well. Like, I'm pretty sure it was free for NeoVim. And so they're like, and we've fixed real things with that and also sent like upstream patches to Vim. And like overall, that's like helpful. Right. But I in general, I don't think Coverity, I don't, I don't think NeoVim gets a deadline for

when this is to fix. And like a disclosure timeline of when it's going to get. published, right? It's like, it's a private, you log into this and view a dashboard. Yeah. Right. And then also there's like some severity levels and some other stuff like that where you could at least try and decide if it's even worth fixing or it's like, okay, sure, this happens when X, Y, Z thing happens, whatever, it's not actually like a big deal. So, you know, so it's like, it is valuable. It's very valuable to get. It helps you fix things and like keep the project in a in a better state and less you know big hacks but yeah it does

feel i can see how like with this timeline pressure applied to it and like there's no like cool you're donating this thing to us to give it to us and stuff and that's nice and everything but it it also does apply real pressure on maintainers and like projects and if they don't close i thought i was reading from them like if they don't close enough of these they get like dinged in like other like ratings of stuff like how secure their project is or something of just like hey if you have a bunch of CVEs reported and you don't fix them even if they're like you know not important you get like ding it's somewhere in some universe of like secure software that can run places and people are happy to use I don't know exactly that's I don't have any secure software so like trash you know

nothing written in Ross baby so someone in chat linked how you get accepted into this project and I'm about to submit all of Dax's projects. And so he could start, so he could start tweeting, hey, man, I just go to LLO and bug report. This is insane. Open code is going in there. Open code CVEs left and right. I love this. Beautiful trash.

I'm going to make the PR and link it. That's so good. That's amazing. Yeah. So, I mean, that's pretty much it. It's less of a answer, less of a, wow, crazy bug. It's more just like a higher level discussion about like, you know, it is a good thing that people are finding bugs of software. That is a naturally good thing. But like, then what, right? Like, what are you supposed to do if you are a team of 10 people against giga corporation X? Right? And, you know, I don't know the answer to that at work.

Yeah, I mean, it's already like that at work. If someone reports a bug on my software internally, I'm like, oh, I don't want to fix that right now. You know what I mean? You know what I mean? It depends on the severity. Like in this case, it was like, like you said, it was like such a niche bug. So if someone reports that, I'm like, you know, make a job. ticket and then maybe I'll open Jira next year, you know. Josh, you said you admitted last week you'd never open Jira in your lifetime. I said maybe if I opened up Jira next year. Oh, okay, all right. Yeah, I mean, that is one, one, like, human factor that I think a lot of people tend to forget. Like, maybe the FFMPEG guy, I don't know where he's worked or whatever. But, like, even where I work, like, large company, right?

If we, on my team, submit a bug to the engineers, like, they have to reallocate resources to fix that bug. And, like, some bugs be submitted, they're like, hey, that's going to take an entire quarter to deal with. Like, we don't have the people right now. It's going to... I assume that's the minimum at Microsoft. What's that?

I thought that's a minimum at any big company, right? Like Amazon, it's just like, no matter where you... Amazon, anybody. Yeah, 100%. I was speaking of Microsoft. Hold on a minute. I had a meme. Give me a second.

Yeah. Yeah, it's just like no matter where... Yeah, there you go. That's the one I was looking at. Yeah. Dude, it's so good. I love this. And what's funny is that, like, it wasn't this big.

It started as a single article and as more bugs... People kept us appending to the meme. Oh, it's so good, dude. Oh, that's funny. Yeah, anyway. So, I mean. At these big corporations, because I know at Netflix, we'd have, it depends on who submits it, too.

And so I think this one thing that's really important here is that some, or at least I think something that can happen is if Google does submit these things and put a lot of pressure on people, also the value of the project does go down. Because now when I hear someone making this kind of submission, I'm like, oh, this isn't that important. because one time I got a bug from Reed Hastings himself saying, hey, there needs to be a vignette on this thing. And so it's just like, you know how fast that got fixed?

It got fixed way, way faster when your VP's telling you what the big man said upstairs. A vignette? Are you kidding? I got to fix that yesterday. But if a company keeps on submitting things and they kind of tarnish their name, people may not take the things they say as seriously. And so there is kind of like a, there's a whole problem there. The other thing that, which I think is a completely re-reacted,

reasonable, like, complaint from FFMPEG team and more broadly in, like, open source generally is just like, it also takes time not only to do the fix, but to do the triage, right? It's like is oftentimes lost, like getting a bug report is not free. Um, so there is something, I don't know how to wait that, right? Like, I would still probably on average, I guess, prefer to know what vulnerabilities exist in the software I'm maintaining versus not. But also like, there's just, only so many hours in the day, bro. There's just like, it's only so many hours. And if I'm going to spend all of it on triaging Big Co, uh, threat vector level low sent to me, like, I'm going to be sad, you know, I'm just going to not want to work anymore or do anything. And like that, yeah. Yeah.

I don't know. Yeah, it's tough, man. Interesting. Interesting. You know, this kind of flows nicely into, uh, something we can talk about here, for a moment here. What a transition, thank you.

I'm going to adopt the transition and we've got transition. I'll just end the episode. I'm done after. Boot up the day. Fibrearer errors on my screen. Terminal coffee and hair.