CyberWire Daily - The secrets behind Docker. [Research Saturday]

Episode Date: April 9, 2022

Alon Zahavi from CyberArk, joins Dave Bittner on this episode to discuss CyberArk's work in conjunction with Patch Tuesday. CyberArk published about how Docker inadvertently created a new vulnerability and what happens when it's exploited. CyberArk's research concluded that an attacker may execute files with capabilities or setuid files in order to escalate its privileges up to root level. CyberArk found the new vuln in some of Microsoft’s Docker images, caused by misuse of Linux capabilities, a powerful additional layer of security that gives admins the ability to assign capabilities and privileges to processes and files in the Linux system The research can be found here: How Docker Made Me More Capable and the Host Less Secure Learn more about your ad choices. Visit megaphone.fm/adchoices

You're listening to the Cyber Wire Network, powered by N2K. data products platform comes in. With Domo, you can channel AI and data into innovative uses that deliver measurable impact. Secure AI agents connect, prepare, and automate your data workflows, helping you gain insights, receive alerts, and act with ease through guided apps tailored to your role. Data is hard. Domo is easy. Learn more at ai.domo.com. That's ai.domo.com. Hello, everyone, and welcome to the CyberWire's Research Saturday. I'm Dave Bittner, and this is our weekly conversation with researchers and analysts tracking down threats and vulnerabilities,

solving some of the hard problems of protecting ourselves in a rapidly evolving cyberspace. Thanks for joining us. The research started as Linux capabilities research and not Docker. But as I went forward with it, I found that Docker had vulnerability and Microsoft had one too. That's Alon Zahavi. He's a security researcher at CyberArk. That's Alon Zahavi. He's a security researcher at CyberArk. The research we're discussing today is titled, How Docker Made Me More Capable and the Host Less Secure. And now, a message from our sponsor, Zscaler, the leader in cloud security. Enterprises have spent billions of dollars on firewalls and VPNs,

yet breaches continue to rise by an 18% year-over-year increase in ransomware attacks and a $75 million record payout in 2024. These traditional security tools expand your attack surface with public-facing IPs that are exploited by bad actors more easily than ever with AI tools. It's time to rethink your security. Zscaler Zero Trust plus AI stops attackers by hiding your attack surface, making apps and IPs invisible, eliminating lateral movement, connecting users only to specific apps, not the entire

network, continuously verifying every request based on identity and context, simplifying security management with AI-powered automation, and detecting threats using AI to analyze over 500 billion daily transactions. Hackers can't attack what they can't see. Protect your organization with Zscaler Zero Trust and AI. Learn more at zscaler.com security. With both of those vulnerabilities, I succeeded at escalation of privileges in a Linux machine.

Well, let's start with Linux itself then. Can you describe for us what are the capabilities here and how do they lead us into a bit of trouble? into a bit of trouble. Traditionally, Linux had two kinds of processes. Processes that ran with root permissions and processes without root permissions. Since Linux kernel version 2.2, those permissions were divided into units called capabilities. And each capability granted the process

the ability to perform certain privilege operations. So let's take, for example, the mount syscall. So before capabilities, only root could have mount new images to the file system. Right now, in order to mount this image, a process will need to have the cap sysadmin capability. Even if it's not root, it can be any user with any UID. And if it's got this capability, the sysadmin capability, it can mount every image to any part of the file system.

And so what does that mean for us in terms of this vulnerability that you discovered? found, if you had GDB, which is a debugger in Linux with the capability of C-speed trace, you can attach yourself as a debugger to any process, even if it's high privileged process. If you had

GDB, if you had access to execute this GDB with this capability, you can do practically anything. You can attach yourself to a root process, which has by default all the capabilities in Linux, and just do anything your mind limits you to. Now, I mean, is this a fundamental flaw in the way that Linux handles this?

Or is this a particular thing that you've discovered here? Linux capabilities, as I said, it's there from Linux 2.2, which is 1999. It's been there for a long time. But only recently with the development of cloud and containers that we see this mechanism started to be used in case of the things I found it's been misused by mistake by some vendors. But yeah, every time process asks to perform privileged tasks, the kernel itself checks if this process has the capabilities to do so. If not, it's just

throwing an error and said you can't do it. Permission error. But if you do have this capability, it just moves on. It's check if you have it, you can do it. If you haven't got it, you can't do it. So how does this affect Docker containers? Okay, so in Docker containers, by default, you get limited capabilities. Because if you had all the capabilities, if the root user from in the container had all the capabilities, it was also the route outside the container.

So you can escape it like if there wasn't any container jail. So in this case, it didn't have the ability to escape a container, had the ability to escape a container. But because of how Docker managed file permissions at the version that I checked, any user, unprivileged user on the host, not the container, on the host, could have access to files that are saved inside the container. So in theory, an attacker that had both access on the host

and the container itself could have done anything he wants. He could have crafted any executable inside the container and give it any capabilities he wants, or even set UID capability, and then move to the host, run it from there, and become root, become the most powerful user in Linux machine. Part of what I find really interesting in this research you published is you walk us through the process of how you discovered this.

Can you share that story with us? Yeah, of course. So this research started as Linux capabilities. I wanted to find misconfiguration or misimplementation, to be more exact, inside the Linux kernel in order to bypass those checks, those capabilities checks. But as I went forward with the research, I found that the implementation was kind of straightforward.

If I talk in code, it was only an if statement. If you got it, you got it. If not, you don't got it. So then I went to search who uses this mechanism. And I found that Docker uses it a lot. Then I scanned over 2,500 Docker images from Docker Hub and found that Microsoft released few images that had that GDB file with file capabilities, which gives any user that executes this GDB the capability to attach itself to any process.

Again, even root processes. And that was the first vulnerability I found in this research. Then in order to execute this GDB, we needed the permissions, you know, classic Linux file permissions. And after looking at the files and directories that Docker Engine produce when you pull an image from Docker Hub,

I saw that one of the directories where the image is saved allows any user to execute any file from within this directory. any user to execute any file from within this directory. And again, then I took the image I downloaded from their website, the Microsoft image, and using low privileged user on the host, all I had to do is just find the path and then execute the file. From then on,

I was root for any... I was root, actually. That's it. Yeah. That's enough, right? Yeah, I think that's enough. It's kind of everything. It's like the Holy Grail for Linux attackers. Right, and I'm imagining you sitting there at your workstation and kind of your eyes opening up and wondering to yourself, wait a minute, is this really what I think it is?

It's exactly as you said. I worked and I tried to execute it and I was like, nah, it shouldn't work. No way. And then I just press enter and it worked. And I was sky high. I went to my colleagues and said, oh my God, oh my God, I found it it that was what i was searching for right so is this a matter that you sort of alluded to this earlier that you know this feature was was introduced into linux back in the very late 90s was it kind of sitting around because um you know back in the 90s, nobody was thinking about cloud environments the way that we do today? I think that it was lying around

because people didn't want to use it. You know, it messes things up. It's not easy as either you privileged or you're not. This kind of mechanism gives you the ability to create a least privileged environment. And as I said, it messes things up. And when you talk about cloud and containers, you have to do it. When you're using a container which has the same kernel as the host, you have to do it. And if you do not do it,

the attacker will have easy attack surface. So that is why, in my opinion, clouds and containers uses Linux capabilities more than the traditional Linux machines. People should have more thinking about this mechanism so people will use it more. Because if more people will use it in its correct way, the machine will be much more secure.

Because, for example, take the ping command, okay? It's not like this anymore because they changed it, but it used to be a setuid binary. So every vulnerability you had in a ping, which all it does is send an ICMP packet, okay? If you had a vulnerability there, you became root. But with capabilities, if you found a vulnerability in ping,

all you had is like, you can send more ICMP. You can build a raw socket and that's it. You will become root. You will have more privileges, but you're still not root. You're still not the most powerful user in Linux. I see. You're limited in what you can do within ping itself. Yeah.

And that's not the case here. No. Again, the GDB, which is again a debugger with the capability to attach itself to root processes, it gives you the ability to, again, to attach yourself to root processes and change the stack and change the memory and change the code flow, the running flow. You can do anything.

You can inject shellcodes and just change the register that points to the next instruction and just run this shellcode. So that's why it's so dangerous to have this capability on that executable. So where do we stand now in terms of Docker itself? They've updated this where where are we yeah so the file permissions that i talked about regarding to to the docker vulnerability it was fixed uh i think a few months

ago a few months ago so that's fixed okay but that's just one vulnerability. So Microsoft fixed their vulnerability also. But it doesn't mean that this escalation of privileges is not there anymore, okay? So Microsoft and Docker did what they had to do. But it's now the task for the IT teams and the system administrators to update Docker engine and to update the images. And they need to scan their machines, their servers,

so they know for a fact that these kind of vulnerabilities are not there anymore. Is there anything to be done on the Linux level itself of limiting Linux's ability to do this? I don't think so, no, because this mechanism is supposed to make things more secure. And when you use it right, it does make it more secure. It's right for everything. You know, if you misuse something, it will be dangerous. But when you use it correctly, it's great, really. Our thanks to Alon Zahavi from CyberArk for joining us.

The research is titled, How Docker Made Me More Capable and the Host Less Secure. We'll have a link in the show notes. Worldwide. ThreatLocker is a full suite of solutions designed to give you total control, stopping unauthorized applications, securing sensitive data, and ensuring your organization runs smoothly and securely. Visit ThreatLocker.com today to see how a default deny approach can keep your company safe and compliant. safe and compliant. Thanks for listening. We'll see you back here next week.