CyberWire Daily - ChatGPT grants malicious wishes? [Research Saturday]

Episode Date: March 18, 2023

Bar Block, Threat Intelligence Researcher at Deep Instinct, joins Dave to discuss their work on "ChatGPT and Malware - Making Your Malicious Wishes Come True." Deep Instinct goes into depth on just how dangerous ChatGPT can be in the wrong hands as well as how artificial intelligence is better at creating malware than providing ways to detect it. Researchers go on to explain how the AI app can be used in the wrong hands saying "Examples of malicious content created by the AI tool, such as phishing messages, information stealers, and encryption software, have all been shared online." The research can be found here: ChatGPT and Malware: Making Your Malicious Wishes Come True Learn more about your ad choices. Visit megaphone.fm/adchoices

You're listening to the Cyber Wire Network, powered by N2K. of you, I was concerned about my data being sold by data brokers. So I decided to try Delete.me. I have to say, Delete.me is a game changer. Within days of signing up, they started removing my personal information from hundreds of data brokers. I finally have peace of mind knowing my data privacy is protected. Delete.me's team does all the work for you with detailed reports so you know exactly what's been done. Take control of your data and keep your private life Thank you. Hello, everyone, and welcome to the CyberWire's Research Saturday. I'm Dave Bittner, and this is our weekly conversation with researchers and analysts tracking down the threats and vulnerabilities, solving some of the hard problems of protecting ourselves in a rapidly

evolving cyberspace. Thanks for joining us. As you probably know, ChatGPT has made a lot of waves since its release. So we wanted to see what we can do with it, whether it's phishing emails or info stealers and stuff like that. So we wanted to see it for ourselves. That's Bar Block, threat intelligence researcher at Deep Instinct. We're discussing their research titled Chat GPT and Malware, making your malicious wishes come true. So you start off here in your research trying to get ChatGPT to write a keylogger for you.

Can we walk through that together? How did you begin? Well, at the beginning, I just simply asked them, write the keylogger. It refused, and it gave a message that keylogging is wrong, malware is bad, stuff like that. And then I thought, okay, so I won't ask it for a keylogger. I will just describe to it what I want the program to do, which is keylogging, but without saying the word keylogger. And it worked. And what was the output that it provided for you? Well, it provided a keylogger in a Go.

That's the language I asked it to write a keylogger in a Go. That's the language I asked it to write a keylogger in. Just a simple program, just records keystrokes, saves them to a file. Later on, I asked it to add a function that can send that file using FTP to a remote location. And that's what I did. And so with your success there, you moved up a level and you asked it to create some ransomware. Take us through how that worked. Well, like before, I asked it to write a ransomware. It refused.

So I just described what the program should do. I asked him to make a program that iterates over directories and subdirectories, encrypts all the files in these directories, and puts a text file in the directory telling with a simple message, which later on I changed to a more malicious one. But it just put a simple message in it. And, well, that was it. Yeah, and it handled the encryption and everything. Yeah.

And this code ran fine. It did what you asked it to do. Yeah, well, I had to add two imports, which Chagipiti omitted in purpose because I know it wasn't, I think it wasn't purpose because it wrote me exactly which imports were missing. So it knew what was needed for the program to run properly. So I just added them and it ran okay. run properly. So I just added them and it ran okay. And it was even able to bypass most of the security products on VirusTotal. Yeah, that's a really fascinating part of this

story here is that you took the results that ChatGPT generated and you ran it through VirusTotal. And what happened when you did that? What were the results there? Well, I think one of the samples, I compiled it to both P42 and P64 versions. One of the versions got three detections. The other got four detections. These are very low numbers because there are like 70 vendors on Viti.

And that's what was detected. I assume that it was because I used Go, because Go is an uncommon language. And that's why I chose it to begin with. And also the encryption was quite simple. It used AES, which is encryption that isn't really used these days in ransomwares. So the simplicity and the use of Go are probably the reasons that got such a low detection rate. And now, a message from our sponsor, Zscaler, the leader in cloud security.

Enterprises have spent billions of dollars on firewalls and VPNs, yet breaches continue to rise by an 18% year-over-year increase in ransomware attacks and a $75 million record payout in 2024. These traditional security tools expand your attack surface with public-facing IPs that are exploited by bad actors more easily than ever with AI tools. It's time to rethink your security. Zscaler Zero Trust plus AI stops attackers by hiding your attack surface, making apps and IPs invisible, eliminating lateral movement, connecting users only to specific apps, not the entire network, continuously verifying every request based on identity and context, on identity and context, simplifying security management with AI-powered automation,

and detecting threats using AI to analyze over 500 billion daily transactions. Hackers can't attack what they can't see. Protect your organization with Zscaler Zero Trust and AI. Learn more at zscaler.com

slash security. You mentioned that the code that ChatGPT had provided to you was almost complete, but there were a couple things you had to add there. Can you explain to us exactly what you think is going on there? Well, I think they omitted these parts by design because they just didn't want to, maybe somehow, I can't be sure of that, they knew that this program may be used for malicious purposes,

so they omitted some of the imports by design, and I know they knew exactly which imports were missing because right after ChatGPT provided me with the ransomware code, it added a message that said that to run this program, I needed two more imports and said which ones. So it knew exactly what it needed to do. It probably just didn't want to supply a working ransomware. Yeah, that's fascinating.

It's like it's saying, you know, here's, I don't know, here's the gun that I built you. By the way, you're going to need some gunpowder. Yeah, something like that. Now, a point that you all make here is that you can use this to help defend against malware as well. And you set this at generating some Yaro rules. What happened there? Well, I asked Chaji Piti to generate Yaro rules for specific MITRE techniques. I gave him the technique ID and it generated a rule.

The rule was very general and it had, if I had used it in an actual environment, it would generate lots of false positives. So obviously it wasn't a good rule. Then I tried to ask the bot to make it less generic, general, and make it generate less false positives. Then it was too specific. But the interesting part was that even though it wasn't as very good at writing YAR rules, it was very good in writing programs that can use the technique I provided it with, the micro technique, and bypass the rules that it wrote itself.

the rules that it wrote itself. So is this a matter of just referencing the YAR rules that it created and saying to it, write something that will bypass the rules that are right above? Yeah, something like that. But yeah, you have to tell him, use that technique and bypass these rules. And then when I asked it to write a rule that can detect the malware it provided me with, it couldn't do that. It just wrote something very, very generic again and didn't really supply with a good rule. It's fascinating to me that it seems as though they've tried to build in

some prevention here but it's relatively easy to work your way around that yeah it's really quite easy all you have to do is like rephrase your request I know it has more

defenses that are not really, not necessarily cyber related, but just, you know, related to like topics it doesn't want to address to. Like everything related to religion and race, things like that. But you can tell it that you can ask it to play a role of another entity. And that entity has to do whatever you tell it to, and then it can do many things that usually Chadi Padi won't do. What's your perception of this as a tool for folks in the business that you're in, doing research and analysis, is there real value here?

There can be a real value here. As I said before, the YAR rules that I was provided with weren't that good, but maybe with more training or giving more examples, it could generate better results. We can also use it to create malware and try to ourselves to defend against it, not using ChagPT, because right now it seems that it's not really good at the defensive side of things. And maybe with some more work,

this Chagibity can even be integrated into SIM systems to provide SOC analysts with more information about things that are going on in their networks. Do you see this as being a tool that could be a time saver for you? In some aspects, it can. For example, if someone is trying to create a malware to try to bypass its own company securities for pen testing assignments, they can use ChatGPT to do that. They can also use it to try to write basic URLs

and try to improve them themselves. And of course, like finding examples for malware that you may have a hard time to find by yourself on Viti, on VirusTotal or something. You can just ask, try to make it, write it and try to see how you can defend against it yourself. So when a real

malware like that tries to get into your network, you will be safe. Wishes come true. We'll have a link in the show notes. Cyber threats are evolving every second, and staying ahead is more than just a challenge. It's a necessity. That's why we're thrilled to partner with ThreatLocker, a cybersecurity solution trusted by businesses worldwide. ThreatLocker is a cybersecurity solution trusted by businesses worldwide.

ThreatLocker is a full suite of solutions designed to give you total control, stopping unauthorized applications, securing sensitive data, and ensuring your organization runs smoothly and securely. Visit ThreatLocker.com today to see how a default-deny approach can keep your company safe and compliant. Saturday podcast is a production of N2K Networks, proudly produced in Maryland out of the startup studios of DataTribe, where they're co-building the next generation of cybersecurity teams and technologies. This episode was produced by Liz Urban and senior producer Jennifer Iben. Our mixer is Elliot Peltzman. Our executive editor is Peter Kilby, and I'm Dave Bittner. Thanks for listening.