CyberWire Daily - Discovering ChaosDB, a critical vulnerability in the CosmosDB. [Research Saturday]

Episode Date: December 18, 2021

Guests Sagi Tzadik and Nir Ohfeld of cloud security company Wiz join Dave to discuss their research "ChaosDB: How we hacked thousands of Azure customers’ databases." Nearly everything we do online these days runs through applications and databases in the cloud. While leaky storage buckets get a lot of attention, database exposure is the bigger risk for most companies because each one can contain millions or even billions of sensitive records. Every CISO’s nightmare is someone getting their access keys and exfiltrating gigabytes of data in one fell swoop. Database exposures have become alarmingly common in recent years as more companies move to the cloud, and the culprit is usually a misconfiguration in the customer’s environment. In this case, customers were not at fault. The research can be found here: ChaosDB: How we hacked thousands of Azure customers’ databases ChaosDB: How to discover your vulnerable Azure Cosmos DBs and protect them Learn more about your ad choices. Visit megaphone.fm/adchoices

You're listening to the Cyber Wire Network, powered by N2K. of you, I was concerned about my data being sold by data brokers. So I decided to try Delete.me. I have to say, Delete.me is a game changer. Within days of signing up, they started removing my personal information from hundreds of data brokers. I finally have peace of mind knowing my data privacy is protected. Delete.me's team does all the work for you with detailed reports so you know exactly what's been done. Take control of your data and keep your private life Thank you. JoinDeleteMe.com slash N2K and use promo code N2K at checkout. The only way to get 20% off is to go to JoinDeleteMe.com slash N2K and enter code N2K at checkout. That's JoinDeleteMe.com slash N2K, code N2K. Hello everyone and welcome to the CyberWire's Research Saturday. I'm Dave Bittner and this is our weekly conversation with researchers and analysts

tracking down threats and vulnerabilities, solving some of the hard problems of protecting ourselves in a rapidly evolving cyberspace. Thanks for joining us. We started playing with the service, started turning on features to see what can be misconfigured. And then we noticed some very interesting feature named Jupyter Notebook. Our guests this week are Nir Ofeld and Sagi Zadig from cloud security company Wiz. The research we're discussing today is titled ChaosDB, how we hacked thousands of Azure customers' databases. Full disclosure, Microsoft is a CyberWire sponsor.

And now, a message from our sponsor, Zscaler, the leader in cloud security. Enterprises have spent billions of dollars on firewalls and VPNs, yet breaches continue to rise by an 18% year-over-year increase in ransomware attacks and a $75 million record payout in 2024. These traditional security tools expand your attack surface with public-facing IPs that are exploited by bad actors more easily than ever with AI tools. It's time to rethink your security. Zscaler Zero Trust Plus AI stops attackers by hiding your attack surface, making apps and IPs invisible, eliminating lateral movement,

connecting users only to specific apps, not the entire network, continuously verifying every request based on identity and context, simplifying security management with AI-powered automation, Thank you. Learn more at zscaler.com slash security. Part of our job as security researchers is to find out what customers are using in their cloud environments and find misconfigurations that can happen in these services so that our product will be able to alert about such misconfigurations. There are a lot of users that are using Cosmos DB, which is a managed database solution offered by Azure.

And these customers are using the Cosmos DB as their database solution. So if they want to store some information in the cloud, they use Cosmos DB for that. We thought that misconfigurations in this service could have a lot of impact. misconfigurations in this service could have a lot of impact. So we wanted to find them so that our product will be able to alert regarding such misconfigurations. And we started playing with the service, started turning on features to see what can be misconfigured. And then we noticed some very interesting feature named Jupyter Notebook. Jup. Jupiter Notebook is an independent product, not Cosmos DB related, that lets you represent your data using live code. So you can write Python code or in Cosmos DB, you can also write C-sharp code to represent the data that you have

in visual ways. And we found that because we were already familiar with Jupyter Notebook, we knew that it lets you execute your own code. So we wanted to see which code we can execute and in what environment. We executed the ID command in our Jupyter Notebook

and we found out that we are running as a low-privileged user. And we started the research from that point. So we were originally trying to find misconfigurations, but we got very curious regarding this feature and the environment we are running in. So we thought it'd be cool to explore it a little bit. Well, let's walk through that together then. I mean, you were able to access some of the primary keys in Cosmos DB? Yeah.

By exploiting a series of misconfigurations, we actually were able to authenticate to the control panel that manages the service cluster. And from that control panel, we are able to list all the databases that are managed in this cluster and their primary keys, which has the password that can be used in order to authenticate to the database and allows for data access and manipulation.

And so when you have that access, I mean, is that the ballgame there? I mean, then you have basically access to everything that's in that database? Yeah, full unrestricted access. We can read data, we can manipulate data, we can insert new records, we can delete existing data. We can do everything that we want in that database

and any other database that we manage to leak the primary key for. So you reached out to Microsoft and alerted them and their response was pretty quick. Yeah, Microsoft responded really quick. We noticed that Microsoft, like only less than 48 hours after our initial report, the vulnerability was already mitigated. And the way that Microsoft chose to mitigate this issue is actually quite interesting. They could have fixed any one of the misconfigurations that we found.

But what Microsoft actually chose to do, it just disabled the Jupyter Notebook feature altogether, meaning that till this day, the Jupyter notebook feature for Cosmos DB does not longer exist. That's actually a pretty funny story. And you can actually see people in the internet complaining that they want to use the Jupyter notebook feature, but it's not there. And I mean, that is a pretty broad brush to paint with, to disable the entire feature. Have there been any indications that Microsoft is working behind the scenes

that maybe it'll return? We don't actually know. Because it's, as you said, behind the scenes. And we didn't see any indications or any Microsoft-published blog post or information stating that the feature is returning anytime soon. So in disabling that feature, is that everything that needs to be done here? Are there any other risks for Cosmos DB users?

So disabling the features essentially mitigated our entry point for the engagement. So the entire engagement was done by abusing the Jupyter notebook container that was set up by Cosmos TV. The result of that engagement was that we were able to leak primary keys. Now, these primary keys are long-lasting secrets, which means that even without the Jupyter notebook feature, we can still use these primary keys until the customers rotate them and revoke them. So we obviously won't use these primary keys because we are not bad actors, but anyone

who exploited this vulnerability prior to us could actually use these primary keys in order to access customers' databases until the customers rotate these secrets. So yeah, the recommendation both from Microsoft and from us is to obviously rotate these secrets and generate new ones. Another thing that we've obtained during the research is credentials that allows us to authenticate to the control panels that manage the service over the internet, meaning that in order to further mitigate this issue from the Microsoft side, they had to regenerate all the credentials for these control panels that we managed to get the credentials for, which is more than 100 control panels.

And they did that. We've seen that they regenerated all the credentials like four days after our initial report. Now, you all recently gave a presentation at Black Hat Europe, and it was titled Security Industry Call to Action. We need a cloud vulnerability database. It's related to this research here. Can you describe that for us? What is your call to action here in terms of a vulnerability database?

Okay, so the issue for Cosmos DB, specifically for Chaos DB, did not actually get a CVE from Microsoft. We think that because there is no CVE, there is no way to reference this vulnerability specifically, it is very hard to talk about this vulnerability. Unless we gave it a name like ChaosDB,

imagine that there was another vulnerability in CosmosDB, you had to refer it to the first vulnerability in CosmosDB, the second vulnerability in CosmosDB. So this is quite frustrating. And another thing is that there is no like a place that organizes all of the vulnerabilities that happen in cloud services like there is

for software solutions, the CVE database. So it's very hard to keep track on which vulnerabilities comes out in which cloud service and there are a lot of cloud services. So this is very hard for the CISO of the company.

And we think that there should be a centralized place where you can see all of the issues that you possibly have in your environment in terms of vulnerabilities that happen and what actions you have to take in order to be mitigated. For example, in the chaos with DV vulnerability, the customer had to rotate their primary key. So we think that there should be a centralized database where it helps you to keep track of all of these things. And this is like the message that we try to pass in the Black Hat

presentation. And who do you suppose would be the best party to run that database? Could it be added to the existing CVE database? So the current CVE specification actually doesn't fit for cloud vulnerabilities. So you have two options. You can create another tracking system, which is one way to go, but it makes the system's life a bit harder because it now has to track CVEs and another tracking system or you can change the current CVE system to include cloud vulnerabilities. And perhaps there are more solutions, this is like the two that we thought about, but there could be

like a better solution and we try to discuss this issue in our Slack group, which has a couple of hundreds of members that really care about cloud vulnerabilities and how they can be managed and fixed. So there is a discussion in this Slack group and we try to think about a general proposal that is good for this. In the Black Hat talk, Shiren alone covered all the things that such a standard should include. But we didn't propose the solution at all, but rather the things that the solution needs to have. And it's like a work in progress.

And as Sagi said, the Slack group is the current place to discuss this issue. So returning to ChaosDB itself, was there any indication that anyone was taking advantage of this, that it was being used in the wild? that anyone was taking advantage of this, that it was being used in the wild? So Microsoft said that there is no indication that someone used this vulnerability. From our point of view, we can't actually confirm their assessment.

As we said before, we urge all customers that didn't regenerate their access keys after KOSDB to regenerate them. Because although we have the Microsoft assessment that this vulnerability wasn't exploited, you can never be like too sure because these secrets are long lasting secrets. And you can see that on a blog

that it could have like been exploited. Our thanks to Nir Ofeld and Sagi Sadek Thank you. show notes. Cyber threats are evolving every second, and staying ahead is more than just a challenge. It's a necessity. That's why we're thrilled to partner with ThreatLocker, the cybersecurity solution trusted by businesses worldwide. ThreatLocker is a full suite of solutions designed to give you total control, stopping unauthorized applications, securing Thank you. The Cyber Wire Research Saturday is proudly produced in Maryland out of the startup studios of DataTribe, where they're co-building the next generation of cybersecurity teams and technologies. Our amazing Cyber Wire team is Elliot Peltzman, Trey Hester, Brandon Karp,

Puru Prakash, Justin Sabey, Tim Nodar, Joe Kerrigan, Carol Terrio, Ben Yellen, Nick Valecki, Gina Johnson, Bennett Moe, Chris Russell, Thanks for listening. We'll see you back here next week.