ColdFusion - Deepfakes - Real Consequences

Starting point is 00:00:00 You are watching Cold Fusion TV. Welcome to another Cold Fusion video. Ten years ago, the word fake would normally be associated with plastic surgery or cheap DVDs. But as the world move more and more online and anyone, anywhere, can create and publish a story or an image, we often have to ask, is this fake? In this episode, we'll take an interesting look at the rise of deep fakes. Is the kind of technology that could potentially save hundreds of thousands of dollars, but also has some real-world consequences.

Starting point is 00:00:38 Let's dive in. But first, I just want to take a second to give a shout out to the sponsor of this video, Audible. So I know most of you guys in the Northern Hemisphere approaching the summer season. You'll be spending more time outdoors at the beach, road-tripping, whatever, but why not make the most of your summer days

Starting point is 00:00:52 by consuming content at the same time? Audible is a great way to do that with an unmatched range of books on every topic, including science and technology. Audible is offering you guys a free audio book with a 30-day trial membership. Just head on to www.audible.com slash coldfusion or text Coldfusion to 500,500 and download a title for free. There used to be a popular saying, the camera never lies.

Starting point is 00:01:18 However, even before Photoshop, this statement was put into question. When this picture was taken originally, it didn't have the Acme logo on it. It was put on digitally, and it can be removed just as easily. Drops of water can be put in a space left by the... erased logo by simply picking up existing drops of water and duplicating them a few centimeters away. The potential for the system goes a lot further than that. Operators say they can take a group photograph for instance and rearrange the people's heads on different shoulders in such perfect detail it would be

Starting point is 00:01:58 impossible to tell a change had been made. It makes you wonder a picture may still be worth a thousand words but what about that old saying the camera never lies. If you have someone on video, then you have the perfect backup for any quote, idea or action that you want to attribute to that person. But this is all about to change thanks to the rise of deep fakes. So what are deep fakes? Deep fakes are altered videos usually of famous people produced by neural networks. It could be superimposing a face onto a body so it looks like they're doing something that they never did. Or you can take some speech and then alter the content making the face movements match the new audio that you've put in.

Starting point is 00:02:41 Even before deepfakes, you've probably seen the basic system at work. It's what powers those goofy Snapchat filters where you can have the googly eyes or the demon face. The tech behind this, face detection, is not exactly a new technology. Even old digital cameras had this feature. In 2001, Paul Viola and Michael Jones proposed a real-time face detection system, now known as the Viola Jones Object Detection Framework. Basically, this framework allows machines to easily detect faces using the differences and brightness between pixels. Today, we still use this basic premise,

Starting point is 00:03:14 but there are many new steps that have been added. Together, they combine to create computer vision. The Viola Jones part is used for high-level detection of the basic markers of a face, but Snapchat trained their system on many hundreds of faces that were manually marked with points to show the borders of lips, eyes, nose and face. The trained application can then take a point mask and shift it to match your individual face, based on the data that it's getting from your camera at 24 frames per second. This is all so you can keep those dog ears in place or do that face swap. Snapchat's technology was mostly built on the experience gained by a Ukrainian company, Luxury, in 2015, and it cost them $150 million to acquire. So in three short years,

Starting point is 00:03:55 things have advanced rapidly. We know that film studios have been able to swap faces for years. Oliver Reed did it for some scenes in Gladiator, and a young Kerry Fisher appeared again in Star Wars Rogue One. But for these, the process is This is long and expensive. Here's how they did it for the Grand Moff Tarcan character in Star Wars. The process we would take to create a shot like this. The first thing we do is we shoot the live action plate photography. This is with Guy Henry as our performer on the set.

Starting point is 00:04:21 And he's dressed in full costume. He has what we call a head-mounted camera rig which is designed solely for capturing his facial performance. Charmed to the last. This is the earliest test. It's the first time we ever saw Guy's motion transferred onto Guy's model and then put onto Tarkin's first like an early likeness. The problem is Cushing's performance and Henry's performance didn't always match.

Starting point is 00:04:46 That required painstaking, sometimes frame by frame adjustments, constantly refining the most subtle details you can imagine. This method took 18 months and would have cost a small fortune. And in just my opinion, the results are within the realm of a video game cutscene. Today though, free apps like FACAB make CGI face mapping a simple tool that almost anyone can operate. To drive this point home, here's a side-by-side video comparison of Princess Leia in Rogue One. One sequence was painstakingly made at a cost likely in the hundreds of thousands,

Starting point is 00:05:17 with the use of an insanely expensive computer. The other one was done in 30 minutes by an average guy on an average computer for free. Yes, there's obviously differences, but honestly, are those differences worth hundreds of thousands of dollars? The previous scene was done with a program called fake app, and here's how it works. Say you want to swap the face of Lewis Lane in the Superman film with the face of Nicholas Cage. The first step is to select your source video, then find a whole range of images of Amy Adams, who acted Lewis Lane, and also a whole range of images of Nicholas Cage. All of these images together serve as the AI's training data. You don't even have to manually download the images one by one.

Starting point is 00:05:56 The app has an automatic script that downloads all of the images at once for you. The second step is to get rid of the parts of the images that we don't want, like objects in the scene. When this is done, it's easier for face detection. The third step is just to let the AI do its magic. A neural network model gets to work learning how to recreate a given face from the images it has. Here's a snapshot example from within fake app. The first six columns show face A being transformed into face B and vice versa in the last six columns. Within each group of three images, the left most is the original image, the middle image is the model trying to redraw the original image, and the right most is the predicted transformation.

Starting point is 00:06:35 The network then outputs the score detailing the amount of error there is within the transformations. When the score is sufficiently low, congratulations, you have a good quality deep fate. Now it's possible to map one face until another face in real-time video using just images. A video is just a collection of rapidly changing images at around 24 to 30 times a second, so this makes sense when you think about it. Technology like this cuts down the cost and effort by many orders of magnitude, putting powerful tools in the hands of the everyday person. But what about the negatives?

Starting point is 00:07:11 Unsurprisingly, there's been a lot of fake adult content created, where existing porn scenes have celebrity faces and post. All you need to do is find an actress with a similar build and then most of the work is done for you by the algorithms. The adult sector was one of the biggest driving forces behind the recent surge of fake videos. However, arguably the biggest danger we face is in politics. Politicians are on camera a lot,

Starting point is 00:07:32 often in quite a fixed position, like standing at a podium or sitting for an interview. This makes them incredibly easy subjects for deep fakes. All you need to do is record any actor making the facial movements you want, and then you could map these onto a politician, altering the way they react or even what they say. And you might be thinking, so you'll still need a really good impersonator to match the voice well.

Starting point is 00:07:53 Well, not for long. At an Adobe conference in 2016, Zihu Jin introduced Voko. This tool was basically Photoshop for audio. He uses a learning algorithm to analyze the speech patterns and convert it into text. With just 40 minutes of speech, it will have examples of almost every sound of that voice in that language. So to recreate the new vocal audio, all you need to do is type. We can just type the word, dogs here. And...

Starting point is 00:08:25 And I kiss my wife and my dogs. Here's more, here's more. We can actually type something that's not here. So let's remove the word my hair and just type the word Jordan. And I kiss Jordan and my dogs. We're not just going to do with words. We can actually type small phrases. So let's say, okay, so we remove those words and we do three times.

Starting point is 00:09:04 Oh. And play back. And I kiss Jordan three times. Recently, researchers at the University of Washington managed to convincingly make Obama say anything. Their AI managed to actually specifically learn how Obama's mouth moved. The heart of our method is a recurrent neural network that transforms input audio to a time-vary mouth shape. Now, most of us don't get our healthcare through the marketplace. the marketplace.

Starting point is 00:09:40 We get it through our job or through Medicare or Medicaid. Then we synthesize mouth texture. And what you should know is that thanks to the Affordable Care Act, your coverage is better today than it was before. Next, we enhance details and teeth. Now have free preventive care. There are no more annual or lifetime limits on essential health care. Finally, we blend the mouth texture onto a retime target video and match the pose.

Starting point is 00:10:05 Women can get free checkups and you can't get charged more just for being a woman. Young people can stay on a parent's plan until they turn 26 and the infrastructure that creates good new jobs. Not to mention the job training that helps folks earn new skills. We can even create an Obama video from voice impressionists. Here we go. President Barack Obama, when you're giving a speech, make sure you use a lot of pauses and speak in a very weird timbre. Up and down, down and up. So thinking about it, if you can make a politician say anything you want, think about the impact that this could have.

Starting point is 00:10:47 So in a broader sense, if these videos become commonplace, perhaps it could give some ground for politicians to actually deny something that they actually said. And then it becomes a question of how do you even trust any evidence given? And this leads us to our final question. How do you prove that a video is real? It's extremely difficult. If you can get a fake video in the raw format that it was uploaded in, there are telltale signs that you can find. Every digital recording device has its own unique algorithms that decide what information is kept. You can't store every pixel of every frame. It's just too much data.

Starting point is 00:11:18 So cameras batch little groups together if they're very close in colour, for example. And this acts as a sort of fingerprint for the camera model, so an expert will be able to tell where this has been altered. But the thing is, once this video has been uploaded and downloaded a few times, it's hard enough to find in the first place, but it'll be almost impossible once it's travelled around the internet. Sometimes there'll be tell-tale glitches when a face moves in a slightly strange way, or if 3D mask doesn't exactly match up with the movements of the head. But the software will keep getting better, making mistakes harder to find. So, is there any hope? Well, yes, there's a couple of things that may save us in the future. For example, we can rely on AI to fight AI.

Starting point is 00:11:55 That is, an AI that's specifically been built to detect fakes. Let's walk through this idea. If an AI is given a training set of fake and real videos and is told which ones are fake and which ones are real, Perhaps after enough training, it'll be able to tell fake videos with better accuracy than we can. Another solution could be verified videos stored in the blockchain so that they can't be altered, and also we'll know that they're real when they're taken from a particular block in the blockchain. So that's kind of where we stand right now. With great power comes great responsibility.

Starting point is 00:12:26 And deepfakes has a highly positive side, but as you can see there's a negative side as well. So I'm going to pass the question on to you guys. What are your thoughts on deep fakes? Let me know in the comment section below. I can kind of see the maturity level of the comment section dropping already. But anyway, this has been Degogo. You've been watching Cold Fusion. If you just stumbled across this channel, feel free to subscribe.

Starting point is 00:12:46 I'll see you again soon for the next video. Cheers, guys. Have a good one.

ColdFusion - Deepfakes - Real Consequences

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.