Darknet Diaries - Ep 9: The Rise and Fall of Mt. Gox

Episode Date: December 15, 2017

Mt. Gox was the largest bitcoin exchange in the world. It suddenly went offline. What happened?

The CEO of Mt. Gox has released a statement for the first time since the Bitcoin exchange was shut down. This amid speculation that the CEO was in hiding, following reports that an estimated 744,000 Bitcoins worth about $350 million was stolen. Japanese authorities are currently said to be investigating the matter. This is Darknet Diaries. True stories from the dark side of the internet. I'm Jack Recider. This episode is sponsored by Delete Me. I know a bit too much about how scam callers work.

They'll use anything they can find about you online to try to get at your money. And our personal information is all over the place online. Phone numbers, addresses, family members, where you work, what kind of car you drive. It's endless and it's not a fair fight. But I realize I don't need to be fighting this alone anymore. Now I use the help of Delete Me. Delete Me is a subscription service that finds and removes personal information from hundreds of data brokers' websites and continuously works to keep it off. Data brokers hate them because Delete.me makes sure your personal profile is no longer theirs to sell.

I tried it and they immediately got busy scouring the internet for my name and gave me reports on what they found. And then they got busy deleting things. It was great to have someone on my team when it comes to my privacy. Take control of your data and keep your private life private by signing up for Delete Me. Now at a special discount for Darknet Diaries listeners. Today, get 20% off your Delete Me plan when you go to joindeleteme.com slash darknetdiaries and use promo code darknet at checkout. The only way to get 20% off is to go to joindeleteme.com slash darknetdiaries and enter code darknet at checkout. That's joindeleteme.com slash darknetdiaries and use code darknet. Support for this show comes from Black Hills Information Security.

This is a company that does penetration testing, incident response, and active monitoring to help keep businesses secure. I know a few people who work over there, and I can vouch they do very good work. If you want to improve the security of your organization, give them a call. I'm sure they can help. But the founder of the company, John Strand, is a teacher, and he's made it a mission to make Black Hills Information Security world-class

in security training. You can learn things like penetration testing, securing the cloud, breaching the cloud, digital forensics, and so much more. But get this, the whole thing is pay what you can. Black Hills believes that great intro security classes do not need to be expensive, and they are trying to break down barriers to get more people into the security field. And if you decide to pay over $195, you get six months access to the MetaCTF Cyber Range, which is great for practicing your skills and showing them off to potential employers. Head on over to BlackHillsInfosec.com to learn more about what services they offer and find links to their webcasts to get some world-class training. That's BlackHillsInfosec.com. BlackHillsInfosec.com.

What if there was a money system that wasn't tied to any specific country? And what if that money system was both anonymous, where you can't tell who owns the money, and transparent, where anyone can see every transaction? What if that money system was completely digital, where you can't tell who owns the money, and transparent, where anyone can see every transaction. And what if that money system was completely digital, where there wasn't a need to print bills or coins? There is such a money system. It's called Bitcoin. Bitcoin is a decentralized electronic currency. Instead of there being a big data center that handles all the transactions, in Bitcoin world, the transactions are done by crowdsourcing. Anyone can join in on processing Bitcoin transactions. But why should they?

When people use their computers to help process Bitcoin transactions, they get a small amount of Bitcoin in exchange. There are a couple of other neat features of Bitcoin. It's not tied to any particular country, it's anonymous, and all transactions in Bitcoin are public information. When someone gives Bitcoin to another person, anyone can see that that money was moved. You might find all this confusing, but just know this. Bitcoin is digital money, and you can buy it in almost the same way you can buy any foreign currency, through an exchange.

You give the Bitcoin exchange your dollars, and they send you the digital money. You can keep your digital money in the exchange itself or move it to your own Bitcoin wallet, which can be stored on your computer or phone or somewhere else. Bitcoin has been around for less than 10 years. It first started in 2008, but for the first few years, there were no exchanges. In 2010, a web programmer named Jed McCaleb took an interest in Bitcoin, and he had an old domain lying around called mtgox.com, which initially stood for Magic the Gathering Online Exchange. But that project only lasted a few months before he abandoned it. So he reused that domain and used it to start the first Bitcoin exchange called Mt. Gox in July 2010. Being first to market of an emerging technology usually means you're going to be the leader of the market. And that was certainly the case for Mt. Gox.

For the entire life of Mt. Gox, it was the dominant Bitcoin exchange for the world. Even when there were multiple other exchanges, Mt. Gox was still processing 70% of all Bitcoin. They were processing over 100,000 Bitcoins a day, which would well exceed 15 million US dollars. You could go to the Mt. Gox website and buy Bitcoin with US dollars, Japanese yen, British pounds, Russian rubles, and several other currencies. You could also do the opposite. Trade your Bitcoin for any of those currencies, and they'll deposit it into your bank account. Mt. Gox was the go-to place if you wanted to buy or sell Bitcoin. But all that came to a screeching halt in 2014. Suddenly, without warning, Mt. Gox went offline and shut down. Mt. Gox contained 750,000 Bitcoins when they shut down. That's 7% of the Bitcoins in the world, which was worth over 450 million US dollars. A statement

from Mt. Gox initially said the Bitcoins were stolen. So what happened? A series of unfortunate events. The technology behind Bitcoin is fairly complicated. Investigators, police, government agencies, they just don't have the resources to really investigate and figure out what happened to the Bitcoins at Mt. Cox. In fact, a lot of it is over my head too. Hello. So I called someone up. Hi Jack, how are a lot of it is over my head too. Hello. So I called someone up. Hi Jack, how are you doing? Who knows a thing or two about Bitcoin.

My name is Kim Nielsen. I'm actually just a general software consultant, work with software development mostly. But I also do blockchain analysis, or as I sometimes call it, blockchain archaeology, since I mostly look into historical stuff. The blockchain is the public record that stores all Bitcoin transactions. There are over 300,000 transactions a day, and each one of these is stored in a way that

anyone can view them all. Bitcoins are stored in virtual wallets. We can see how much Bitcoin moves around from one wallet to another, but we can't determine who that wallet belongs to. That part is anonymous. I was pretty much a Mt cox customer yes uh i was just using them for trading and whatnot i wasn't really actively observing them while they were up or anything so i was kind of taken off guard when mount cox just went down and i didn't lose that

an insane amount of money but i didn't't take it particularly well and felt like someone needs to investigate this. And I wasn't really prepared to trust that normal law enforcement and the usual actors would investigate it properly, considering what a cutting-edge technical field it is and would they even get the necessary data and whatnot. And then I sort of kept watching the situation going forward from there

with great interest and there started to be leaks coming out with some of the internal Mt. Gox data and suddenly, well, actually maybe someone else can investigate this from the outside. And that's sort of where this all got started.

And I joined forces a little bit with some other guys and cooperated when necessary. And we tried to get hold of as much data as we could to try to figure it out. And after that, it's basically just been me plowing through that and doing technical analysis on it ever since. So for the last three years, Kim has been studying the blockchain and trying to personally piece together what happened to Mt. Gox. He's trying to count how much money went in, how much money went out, and reconcile these numbers. This is no easy task. The data containing all these transactions is almost 40 gigabytes,

and there were hundreds of millions of records. He was trying to go through all of them to try to make sense of what happened to Mt. Gox. And he spent a long time crunching this data and found brilliant ways of searching it faster. And he would also ask the CEO of Mt. Gox directly for additional information. And the CEO would sometimes comply. It also helped that they both lived in Japan.

As you say, one of the things that makes it quite hard to get an overview of what actually happened in Mt. Gox is that there really wasn't any one thing. There was a lot of different things. Some of them are connected. Some of them aren't. So it's hard to get a single satisfying answer to what actually happened. So let's start from the beginning and try to find where Mt. Gox went wrong. Right, so Mt. Gox was founded or reworked as a Bitcoin exchange in 2010. Which was likely ran by a single person at the time, Jed McCaleb.

He was a web developer and personally created the site from scratch. In January of 2011, there was two exploits related to Mt. Gox integration of Liberty Reserve withdrawals. Liberty Reserve was a company Mt. Gox used to transfer money from one person to another. It was a service tied to their backend, which allowed money to be moved around. But Jed made some mistakes when adding the service to his site. And a user would just be able to inject XML to override parameters in the API request sent to Liberty Reserve to get more money than they actually withdrew from Mt. Gox. So hackers were withdrawing more than they should be allowed to withdraw by exploiting this poorly implemented code. But that wasn't the only issue Jed had with Liberty Reserve.

Users type in how much money they want to withdraw, but... The code forgot to check for negative inputs. This had screwy results, allowing users to withdraw money they didn't have. Jed found these bugs and fixed them, but it wasn't until he had already lost $50,000. Around that time, Jed McCaleb and Mark Karpolis started talking. Jed was realizing the site required more time than he could put into it, and Mark showed an interest in digging over Mt. Gox from Jed. Mark was a web programmer from France, but had recently moved to Tokyo, Japan, where Mt. Gox was located.

So Jed basically almost gave it away to Mark Karpolis in that very favorable price, and he basically sold it for almost no upfront money and just getting a cut of the revenue for the next six months, I think. But just before the transfer of ownership to Mark, something terrible happened to Mt. Gox. Someone had broken into the Mt. Gox servers and stole the hot wallet file. A hot wallet is a Bitcoin wallet Mt. Gox used to conduct daily operational trades. This differs from a cold wallet, which is not a Bitcoin wallet Mt. Gox used to conduct daily operational trades. This differs from a cold wallet which is not stored on the Mt. Gox servers but in another location in a much safer place. So the thief took the hot wallet and then transferred all

the Bitcoins that were in it to their own wallet. So that was about 80,000 Bitcoins that disappeared and are actually still sitting untouched on the blockchain to this day. So it's quite possible they've been accidentally destroyed or something. Since these 80,000 Bitcoins had been stolen, there was of course at this point already a shortage. So Mt. Gox was technically already insolvent as soon as Mark Karpolis took it over. I don't think Mt. Gox was ever, in fact, solvent for a single moment for as long as it existed under Mark Karpolis. That is, if all the Mt. Gox users were to try to take all their Bitcoins out of the exchange, there wouldn't be enough for everyone.

So already by the time Mt. Gox was sold a few months later, there was a lot of money missing. So off to a bit of a bad start under his management, you could say. Mark Karpolis is now running Mount Gox from Tokyo, Japan. All ownership is now transferred to him and a few months go by under his management. Mark was in the process of figuring out where the Mount Gox bitcoins should be kept. He had some in the Mount Gox servers in the form of a hot wallet, and he had some in a secure offline location, which is a cold wallet, but he also had some on his personal computer. During this time, it appears that someone was able to get into Mark's own computer, which at this particular point in time appears to have been quite unsecured. It looks like they got it

off of his personal machine from home, and at the time, he was unfortunately keeping 300,000 bitcoins from Mt. Gox on his own unsecured machine. And a thief was just poking around, got lucky and found it and ran away with it. This is a huge number of bitcoins lost, obviously, but it never got any kind of mainstream attention because actually the thief, as it turned out, got nervous, it seems, and offered to give all the bitcoins back in exchange for a small keeper's fee. The thief kept 3,000 bitcoins and gave the remaining 297,000 back to Mark. Probably, if I were to venture a guess, they offered to give the coins back because they hadn't been particularly careful. I don't think it was some master hacker. It was probably just someone poking around, like I said.

And if I had been in their shoes, I probably would have been astonished that it even worked in the first place. Presumably the deal at the time was that they gave the coins back in exchange for not being investigated. So another month goes by. It's now June 2011. Mark had only been running Mt. Gox for about three months.

At some point, someone again got into Mt. Gox's system and were probably able to take a small database dump of the users table that contained accounts and their password hashes and everything. Whoever stole this small database dump were able to also brute force a fair amount of the passwords, including the password to Jed's own admin account. And part of having an admin account on Mt. Gox meant you had access to a small separate page where you can do admin tasks like manipulate account balances and things like that.

So whoever got into this made good use of this little feature and started adding crazy amounts of Bitcoins to new accounts. The hacker was creating Bitcoin out of thin air and selling them as fast as they could. A Bitcoin was worth around $17 at the time. This hacker sold so many Bitcoins, they drove the price all the way down to one cent each. Some people bought thousands of Bitcoins at that price. The market had gone haywire and hit rock bottom.

Then the thief bought back the cheap bitcoin and tried to move the bitcoin to another address. But Mark Karpolis saw this crash happening and shut the servers down. He knew something was very wrong and investigated. He even had help from others to investigate the situation. Mt. Gox remained offline for days. They eventually found how the hacker got in and crashed the market. T-Mark rolled back the system to undo all trades during the time of the crash. Mt. Gox came back online with the price restored back to $17 per Bitcoin. But the hacker wasn't completely empty-handed.

They were also able to get some actual Bitcoins out by inflating balances, trading them for a bit, and just trying to withdraw them. And they got about 2,000 Bitcoins out that way. That is 2,000 Bitcoins that Mt. Gox had to absorb since they were actually transferred out of Mt. Gox. A couple more months go by, and Mark is looking to expand Mt. Gox to be able to do business in Europe.

In late summer 2011, there was this other exchange in Poland called Bitamat that accidentally destroyed their own Bitcoin holdings. I think they had their wallet in the virtual machine and accidentally wiped it or something like that. And that destroyed 17,000 Bitcoins. And at this point, Mt. Gox was sort of looking to expand and have local licenses and whatnot to operate in local markets. And they were trying to get sort of a chance to get into Europe by acquiring Bitomat and whatever company registrations they had and whatnot in exchange for covering this debt, basically. So Mt. Gox absorbed all of Bitomat, including the 17,000 Bitcoin in customer holdings. To be continued... and was surprised by just how much stolen identity data criminals have at their disposal, from credentials to cookies to PII.

Knowing what's putting you and your organization at risk and what to remediate is critical for protecting you and your users from account takeover, session hijacking, and ransomware. SpyCloud exists to disrupt cybercrime, with a mission to end criminals' ability to profit from stolen data. With SpyCloud cloud a leader in identity threat protection you're never in the dark about your company's exposure from third party breaches successful fishes or info stealer infections get your free dark net exposure report

at spycloud.com slash dark net diaries the website is spycloud.com slash darknetdiaries. So at this point, Mark had only been running Mt. Gox for about five months and is already missing $50,000 in cash and a massive 100,000 Bitcoin. While Mark's skill set was mostly in programming, he believed Mt. Gox could be turned around and become a very profitable company. He had every intention on making it a success. He was just in a little over his head. Not another month goes by, and there's another breach.

So yet another incident where someone seems to have gotten into the system, have access to the database. Recall that the June hack as well involved getting information out of the database. Recall that the June hack is well involved, getting information out of the database. This time around, though, there definitely seems to be that the attacker has right access to the database as well, that they can make changes directly to it, and they alter account balances.

They are able to wipe accounts. They try to wipe their tracks as well in this way. And because of that this this incident is a bit harder to track because a lot of the evidence seems to have been wiped and i've sort of reconstructed some of it by basically by sketching out the gaps where there should have been logs but there aren't any or there are there are orphan records in the database that suggest that something has been deleted and whatnot. But probably by the best estimates, something a bit short of 80,000 bitcoins

was taken out of Mt. Gox. Mark did not detect this breach until after the bitcoins were already transferred out of Mt. Gox. So he wasn't able to stop this theft from happening. Mt. Gox is now missing almost 200,000 Bitcoins. In the very next month, another breach occurs. And this one, the biggest one of all. The hot wallet, wallet.deck, got stolen by someone with access to the system.

Once again, a hacker broke into the Mt. Gox server and stole the Bitcoin wallet from Mt. Gox. The thief quickly moved the Bitcoin into their own wallet. But because they had control of the main hot wallet from Mt. Gox, any new deposits into that wallet, the thief would also be able to take those funds too. And for the longest time, Mt. Gox didn't detect any of this activity. This theft went on until at least a good chunk into 2013.

So that's almost a full two years now. By the time it was finally detected and stopped, the thief had been able to siphon 650,000 bitcoins from Mt. Gox. So this is basically the big theft that most people will be sort of thinking of when they talk about the Mt. Gox missing bitcoins. You might be wondering, why didn't Mark encrypt the wallet?

Well, ironically, the feature to encrypt bitcoin wallets was released a few weeks after the theft took place. So there simply wasn't a technology out there to encrypt Bitcoin wallets at the time. You might also wonder why Mt. Gox was target for so many attacks. Bitcoin is basically incentivized hacking in that now that you have things like cryptocurrency, you can actually steal actual money just by hacking computers whereas previously you could just acquire the means of stealing money and whatnot you can you could get data and holding it for ransom and whatnot but with bitcoin and other cryptocurrency you can actually steal the actual money from from digital systems so that increases the risk by a magnitude easily in that it now is a lot more lucrative for hackers to attack your systems and mongox had the it by a magnitude easily in that it now is a lot more lucrative for hackers to attack your

systems and mancox had the it was a double-edged sword in that it pioneered a lot of the market but also since it was the first it it's not like this was some big established company that spent millions developing secure software or anything it It was just an amateur code base that they were then trying to keep patched and somehow handle the load over the years. It was never a professional exchange in the sense that people who deposited money probably thought or wanted it to be.

Even though, by this time, Mt. Gox had lost over 800,000 bitcoins, the problems didn't stop there. Another error in the code time, Mt. Gox had lost over 800,000 bitcoins, the problems didn't stop there. Another error in the codebase of Mt. Gox had a problem processing transactions. So this showed up in the Mt. Gox system as a deposit to a user account, even though there were no new bitcoins coming in. But it was misidentifying those transactions as putting new bitcoins into user accounts. It was a bit under 50 accounts. They got free Bitcoins for about 45,000 Bitcoins in total. So there went another 45,000 Bitcoins.

During that month of October 2011, Mark Karbalis deployed a new wallet system in the Mount Cox that he had programmed himself in an attempt to have a more secure service. But near the end of 2011, his new wallet system had a programming error. This is an instance where he actually had a bug in his code. And that meant that a number of withdrawals actually sent Bitcoins to unspendable addresses, which is the same as destroying Bitcoins. So that lost about 2,500 Bitcoins

that became destroyed because of this. The irony is that this is basically the only real instance of Mark's replacement wallet actually losing any significant funds. Mt. Gox didn't have any more thefts or break-ins since the new wallet software was added. It seems like his improved security was working. But Mark knew he was missing so many Bitcoins, and Mount Gox was insolvent. And he tried to keep all these breaches and thefts under wraps and hidden from the public. He didn't want to ruin the reputation of Mount Gox being the leader of Bitcoin exchanges, so he tried to keep these breaches secret.

Things went okay for Mark and Mt. Gox in 2012. No significant breaches occurred, no big programming errors that resulted in a loss of Bitcoin, and Mt. Gox continued to dominate the market as the largest Bitcoin exchange. It appeared that Mark had finally gotten things under control and was slowly recouping his losses. By mid-2013, Mt. Gox was handling 70% of the world's Bitcoin trades, which was around 150,000 Bitcoins a day. Even though it was the leader of Bitcoin exchanges, there weren't that many employees working for Mt. Gox. Mark did a lot of the programming himself, but he did have a few other developers helping out. It's not clear whether Mark thought he could just handle it himself, or he had a hard time trusting others, or maybe he didn't have the money to hire staff.

In 2013, Mt. Gox wanted to expand into the U.S., so it agreed to use a company called CoinLab to handle all North American transactions. Mt. Gox gave them $5 million to get started with. But for some reason, this deal went bad, and CoinLab filed a lawsuit against Mt. Gox because of a breach of contract. CoinLab was seeking $75 million, and they kept the $5 million that Mt. Gox gave them. And I believe this lawsuit is still going on today. Later in the summer of 2013, the US Department of Homeland Security issued a warrant to seize money from Mt. Gox.

They had been in violation of acting as an unregistered money transmitter in the U.S. In total, DHS seized $5 million from Mt. Gox. It was also discovered that Mark was running a trading bot. I don't know who had the original idea. There was an email leaked where Mark and Jed is discussing it. And Jed is, amongst other things, suggesting that maybe you can make trades on your own exchange to sort of shift this liability back and forth between Bitcoins and fiat money as a way to recover the funds. If you can basically trade the losses back.

Of course, require you to be a trader that can make reliable profits. And Mark doesn't seem to have been a trader that was able to make profits. If anything, he seems to have bought high and sold low. This trading bot he was running was called the Willy bot, and there's a paper that explains it in detail. At the time the paper came out, it was just speculation though. But the speculation proved to be spot on. The way Mark implemented this internal trading mechanism, it actually kept all the logs,

so it's possible to get quite accurate information about what was done or not. It didn't seem to wipe any logs or anything like that. By analyzing the logs and adding up the trades, the spot resulted in Mark losing an additional $50 million and 22,000 bitcoins. In February 2014, Mt. Gox suddenly, without warning, halted all withdrawals from its site. Mark stated they were addressing some security concerns. But a few weeks later, Mount Gox, the largest Bitcoin exchange in the market, shut down entirely. The website went offline and simply returned a blank page. Protests began outside the Tokyo office. Thousands of customers were furious. Mt. Gox had

closed its doors, seemingly holding on to 850,000 bitcoins. But in reality, the Mt. Gox coffers were entirely empty. At the end of February, Mt. Gox had filed for bankruptcy in Tokyo, stating they had lost 850,000 bitcoins, which was worth $470 million. Nobody believed that Mark had lost this many bitcoins. That's 7% of all the bitcoins in the world. People just couldn't grasp how such a large amount of bitcoins could become lost. But as you heard, it was lost. Just not all at once.

The next month, when looking through some of the old Mt. Gox systems that were in use in 2011, Mark Karpolis found a wallet in the system that still had 200,000 bitcoins. This brought the total losses down to 650,000 bitcoins. In August 2013, Tokyo police arrested Mark Karpalis, not for losing the 650,000 bitcoins, but on charges for embezzlement and fraud. Japanese prosecutors accused him of falsely adding bitcoins into the accounts of investors and then moving that to his own wallet. He has continued to plead innocent to these charges. As far as the thieves go who stole the money from Mt. Gox, only one person has been caught and arrested. His name is Alex

Finick and is a Russian national and was arrested in Greece. He was arrested for money laundering, but evidence shows that he had some of the bitcoins that were stolen from Mt. Gox. However, just because he had them doesn't mean he stole them. He was just accused of laundering, not of stealing the Bitcoin. A year after his arrest, Mark Harpalus was released on bail and is currently residing in Tokyo. He's still undergoing the legal process of his arrest and the Mt. Gox bankruptcy. Those 200,000 Bitcoins he found have sat untouched since he found them.

The bankruptcy case restricts him from touching them. But since 2014, when Mt. Gox filed for bankruptcy, the price of Bitcoin has skyrocketed. At the time of this recording, it's now worth over $10,000 per Bitcoin. Mt. Gox owes their creditors $450 million, which is what the Japanese courts have sort of locked in as what he owes. But his 200,000 Bitcoins are now worth $2 billion.

So if he were to pay off all his creditors today, he may then get to walk away with over $1 billion. billion dollars. For sure, the industry and the community has learned a lot since Mt. Gox. Like I said, most of the reason for Mt. Gox being so unprepared to fill its own shoes, so to speak, was because it was the first of its kind. There was no real best practices back in 2011 or anything like that. And as anyone who's been in an initially successful startup can tell you, it can be really hard to keep up with actually investing the time needed to keep your technology up to date. One big thing we've learned from Mt. Gox is that it's not a good idea to leave your Bitcoin on the exchange unless you're actually trading it.

For long-term storage, it's best to keep a Bitcoin wallet on your own computer that you control. This way it can't be stolen from an exchange. But you still need to be careful and protect your computer from thieves though. When determining if an exchange is reliable, check to see if it complies with audits. For Bitcoin exchanges to operate in the US, they must adhere to strict audits, but some exchanges even take it a step further and agree to even more strict audits. It's also good to use an exchange that has a good reputation, and not something that has just popped up yesterday. A lot of lessons can be learned from Mt. Gox. Yeah, it's quite a wild ride.

And it will be interesting to see what happens next. Not just to Bitcoin, but also to Mal Gox and Mark Karbalis. You've been listening to Darknet Diaries. For show notes and links, check out darknetdiaries.com. Music is provided by Pottington Bear. Hey, are you learning to program in Python or want to learn? I've got something for you. I put together a Python cheat sheet,

which is a single page of the most common Python commands. It's a perfect reference guide to print out and have handy for when you need to look up the syntax of a command. I made this for myself when I was learning to program in Python. And when I made it, I ran it by the Python community and got an overwhelming positive response. I even added some of their suggestions to improve it. You can download this Python cheat sheet at darknetdiaries.com.