CyberWire Daily - Essential tools with critical security challenges. [Research Saturday]

Episode Date: August 17, 2024

Snir Ben Shimol from ZEST Security on their work, "How we hacked a cloud production environment by exploiting Terraform providers." In this blog, ZEST discusses the security risks associated with Terraform providers, particularly those from community sources. The research highlights the importance of carefully vetting providers, regular scanning, and following best practices like version pinning to mitigate potential vulnerabilities in cloud infrastructure management. The research can be found here: The hidden risks of Terraform providers Learn more about your ad choices. Visit megaphone.fm/adchoices

You're listening to the Cyber Wire Network, powered by N2K. of you i was concerned about my data being sold by data brokers so i decided to try delete me i have to say delete me is a game changer within days of signing up they started removing my personal information from hundreds of data brokers i finally have peace of mind knowing my data privacy is protected delete me's team does all the work for you with detailed reports so you know exactly Thank you. Hello, everyone, and welcome to the CyberWires Research Saturday. I'm Dave Bittner, and this is our weekly conversation with researchers and analysts tracking down the threats and vulnerabilities, solving some of the hard problems, and protecting ourselves in a rapidly evolving cyberspace. Thanks for joining us. So we're dealing with a lot of cloud security teams and also DevOps teams with securing the

cloud and working with their Terraform and cloud formation and Pulumi environments. By doing that, we're identifying many vulnerabilities and maybe potential malicious providers and add-ons to those systems that the security team have no idea that they're there. That's Sanir Ben-Shemal, CEO at Zest Security. The research we're discussing today is titled, How We Hacked a Cloud Production Environment by Exploiting Terraform Providers. So there is very kind of lack of visibility to those type of areas in kind of DevOps security realm. And while working with those customers, we kind of decided that this is a good opportunity to share our knowledge with the community and kind of make sure that they're all looking into those places. Well, for people who might not be familiar with Terraform,

can you give us a little description of what it's used for and the scope of its capabilities? Yeah, definitely. So Terraform is basically an infrastructure as code platform or tool. It's used primarily by DevOps and platform teams to automate many types of infrastructure tasks in the cloud. Basically, it's the platform that allows you to operate a CICD operation. And the provisioning of cloud resources, for instance, is one of the main use cases for Terraform. This, for instance, is one of the main use cases for Terraform.

I see. Well, let's walk through your discovery process here. I mean, what exactly did you find when it came to vulnerabilities with Terraform? Yeah, so this is not a new vulnerability in Terraform or new vulnerability in Provider, but it's more so an attack path or a potential way to compromise a cloud environment if the specific platform team is using third-party Terraform providers that have known vulnerabilities. So maybe we'll start with just talking a little bit about Terraform providers. So exactly like everybody knows that you have third-party in source code, you know, all the log4j and the solo wins attack kind of put a lot of attention of which type of third-party components

your developers are using to develop their code. And those components can be vulnerable. And this is really kind of well-known and well-covered already when we're talking about developers. But when we're looking on the DevOps side of the house, it's also code, but it's infrastructure as code. And those systems that are really, really sensitive, like Terraform, deploying and provisioning cloud resources also use third parties.

And one of the ways they're using third parties are, it's called Terraform providers. And providers can be responsible for API interactions, exposing resources, interactions with different types of APIs required to create and update and delete various resources. And something we shared in our research is you have basically three types of those third parties, those providers. You have official ones, which are owned and maintained by AshyCorp, which also is the company who created and maintained Terraform. You have partners. So those providers basically own and maintain and created by any type of tech company. And that goes through a partner onboarding process, validation and being vetted by AshiCorp themselves.

So those are kind of like official partners. And then you have the third one is community providers. Those are basically published and maintained by individuals or communities or just, you or just DevOps or software engineers or more like open source told providers. So those are the three types. By looking in on those three types of third parties, we did a breakdown of how many vulnerabilities you have

in official providers, partner providers, and community providers. And that was kind of interesting. And this kind of basically allow us to run this research and talk to a lot of security practitioners and ask them, are you aware of those vulnerabilities in your telephoneform providers? And most of them said no. And I think the main reason was

we don't have a lot of security or offensive security or security write-ups around Terraform system. So it was also, okay, this is what we need to do. We need to share what we know and raise the knowledge around it. Yeah. I mean, it is interesting looking at the research. You have some graphs here that show the percentage of vulnerabilities by the different provider types. And I guess unsurprisingly, it's those community ones that have the highest number of vulnerabilities. But then you also look at the downloads and it appears to me like

it's really these partner-provided ones that see the most downloads. Is that accurate? It really depends. I think when you use providers, most of the time you will lose the ones that Ashiko will provide you and the ones like partner providers. So it's pretty known and it's pretty normal. And the community providers normally is like if you do have some specific project that fits your needs as an engineer, you will tend to use it.

Many of those teams, by the way, the people who are using those type of providers are not security people. They're like DevOps people, which, by the way, from my experience, have more security knowledge than just a normal engineer or software developer. But yeah, they're using more of the official or the partner providers. And also, it's very important to mention, we're not saying that the community is specifically more vulnerable, but we can see they have more vulnerabilities. That actually can be very positive things that the community

is being scanned and being addressed vulnerability. Those are known vulnerabilities. So they identify them and they fix them. So it's actually something that I'm looking at as a positive thing that you're aware of your vulnerabilities and you actually have upgraded versions and fixes for those vulnerabilities. We'll be right back. Do you know the status of your compliance controls right now? Like, right now.

We know that real-time visibility is critical for security, but when it comes to our GRC programs, we rely on point-in-time checks. But get this, more than 8,000 companies like Atlassian and Quora have continuous visibility into their controls with Vanta. Here's the gist. Vanta brings automation to evidence collection across 30 frameworks, like SOC 2 and ISO 27001. They also centralize key workflows like policies, access reviews, and reporting, and helps you get security questionnaires done five times faster with AI.

Now that's a new way to GRC. Get $1,000 off Vanta when you go to vanta.com slash cyber. That's vanta.com slash cyber for $1,000 off. Well, you outline an attack scenario here in the research. Can you walk us through what you've got here? Yeah, so this is basically, we got a lot of questions of like, why do we care? And this attack scenario is basically different type of ways. And when we cover different type of way of potential threat can cause just by not paying attention of which provider you're using in your environment. So let's say that you have an attacker that have some kind of physical or remote access to that specific environment. It cannot really modify anything within the CACD.

You have kind of like a read-only access or view-only access of some of those type of systems. And this is a virtual ground. Those vulnerabilities or misconfiguration can be used for privilege escalation. So if I don't really have the capability to do everything I want, I can basically try to leverage those CI-CD system that usually run and execute code and operating

in very high permissions. I want to go after them. Those are kind of the new crown jewels. So the first thing I will do is like, I will look on which type of providers that specific project is using. I can retrieve that information if I have view only of those files.

Also something very, I think, important to mention that people are not really paying attention to. Some of those Terraform files are being managed in GitHub or GitLab or the Git repositories that tend to be pretty secure and you have some guardrails around them and all the developers can look into it. We can definitely improve there and you have best practices, but it's a pretty hard thing to actually go into the company source code repository.

I hope it is, at least. But some of those, like those terraforms that the infrastructure has been deployed and you have a plain file and you have state files that contains all those

infrastructure code configurations, they're normally not being stored only in the Git, but they're being stored in, let's say, S3 buckets or some kind of other repositories across the cloud. So even if I don't have remote or physical access

to your source code, there are a lot of, again, being part of a lot of incidents, like leading incident response team, you do have a lot of threat actors or insider threat that have access to random S3 buckets or some repository or some places that have those Terraform files.

And those Terraform files, of course, it's not best practice, can have credentials. So you can skip all these fancy attack scenarios that we built and just take those credentials and own the system. But we hope that everyone following the best practices and they don't have secrets in their Terraform.

But you need to remember those that information it's not only if you have access to the source code, it's also only if you have access to the source code, it's also if you potentially have access to those S3 buckets, even if it's read-only. So again, this is kind of interesting, and we want organizations to be more aware of

how they protect their Terraform state files, how they protect their DevOps system. So this is something I really want to mention here. So let's say we have that access, it's read-only. We identify that they're using Tilt-Pilot providers. You have kind of more known vulnerabilities in the community versions. So I'm looking on the community versions providers.

I am identifying a vulnerability. And in this case, I can exploit this vulnerability in order to escalate my privilege and then do what I want to do and take over the cloud. This is basically the flow of the attack. I see. So because you're able to have that view into the plugins that someone's using here, then you can use that as a guide to find the potential vulnerabilities you can exploit. Exactly. And again, those vulnerabilities are non-vulnerabilities.

It's not like a zero-day, but the attack path is interesting because apparently no one is looking into it. That information can be visible to many cloud users and many SaaS applications. So those Terraform files are basically outside of the code repositories. And in most cases, I will say even 90% of the cases, those vulnerable providers are not being popped up during security scan. So if you just scan your infrastructure as code, as a code scanning, you know, application security,

normal SDLC secure development, secure design lifecycle, they're scanning the infrastructure as code and they're mostly finding things like misconfigurations. They're not scanning, most organizations at least, today are not scanning those files for providers' vulnerabilities. And this is where it's a blind spot in many cases in the secure development lifecycle of organization.

That you scan for misconfiguration, you're not scanned for provider vulnerabilities, and you actually use vulnerable models. I'm sorry, not models. And you actually use vulnerable providers, and those vulnerable providers are now part of your cloud production system. And those vulnerabilities can be exploited in various ways. Well, so what are your recommendations here? I mean, how can folks go about mitigating this? Yeah, so first and foremost, again, this blog post is basically to raise an awareness.

And we're calling application security engineers and even DevOps and platform teams to check and do some kind of due diligence before using a provider. They need to check if they're using the latest version. They need to check if those providers have known vulnerabilities that can put themselves in a risky position. Some of those vulnerabilities can be there, but they cannot be exploited, which is fine.

And this is why you just need to be aware and to do those security checks. We recommend to do it as early as possible in the development cycle as well. And the other thing is, even if today the provider is not vulnerable, maybe a new vulnerability can be introduced later on. So always monitor regularly your Terraform plans and state file for any misconfiguration or expected changes. And kind of look into those providers and check that if new vulnerabilities or critical vulnerabilities have been identified. This is very critical. And again, it's kind of a weak spot from what we're seeing right now

because, again, it's not part of actually your own code that you scan and you check for those type of third parties. But on the same time, it's not actually happening later on as well. So it's kind of like that weird blind spot that we have in our processes right now in most organizations. So just make sure that you use the right providers and you vet those providers in a good way. And by the way, something also cool

to mention, if you use a community providers, you do want to make sure that those community projects are being well maintained and managed and being built by kind of a valid community as exactly like any other third party open source that you use, right? It's also, we have some reports of malicious providers, malicious kind of third parties that Terraform can use. So you need to be aware of that as well.

On the same time, also to scan those vulnerabilities as part of the due diligence process before using a provider. Our thanks to Sanir Ben-Shemal from Zest Security for joining us. The research is titled, How We Hacked a Cloud Production Environment by Exploiting Terraform Providers. We'll have a link in the show notes. environment by exploiting Terraform providers. We'll have a link in the show notes. And now a message from Black Cloak. Did you know the easiest way for cyber criminals to bypass your company's defenses is by targeting your executives and their families at home? Black Cloak's award winning digital executive protection platform secures their personal devices, home

networks, and connected lives. Because when executives are compromised at home, your company is at risk. In fact, over one-third of new members discover they've already been breached. Protect your executives and their families 24-7, 365 with Black Cloak. Learn more at blackcloak.io. We'd love to know what you think of this podcast. Your feedback ensures we deliver the insights that keep you a step ahead in the rapidly changing world of cybersecurity. If you like our show, please share a rating and review in your favorite podcast app. Please also fill out the survey in the show notes or send an email to cyberwire at n2k.com. We're privileged that N2K Cyber Wire is part of the daily routine of the most influential leaders and operators in the public and private sector, from the Fortune 500

to many of the world's preeminent intelligence and law enforcement agencies. N2K makes it easy for companies to optimize your biggest investment, your people. We make you smarter about your teams while making your team smarter. Learn how at n2k.com. This episode was produced by Liz Stokes. We're mixed by Elliot Peltzman

and Trey Hester. Our executive producer is Jennifer Iben. Our executive editor is Brandon Karpf. Simone Petrella is our president. Peter Kilby is our publisher. And I'm Dave Bittner. Thanks for listening. We'll see you back here next time. Thank you. With Domo, you can channel AI and data into innovative uses that deliver measurable impact. Secure AI agents connect, prepare, and automate your data workflows,

helping you gain insights, receive alerts, and act with ease through guided apps tailored to your role. Data is hard. Domo is easy. Learn more at ai.domo.com. That's ai.domo.com. That's ai.domo.com.