Advent of Computing - Episode 116 - Monte Carlo

Episode Date: September 10, 2023

It's finally time! In this episode we are looking at the Monte Carlo method, perhaps the first practical computer program that could outpace human capability. The best part: the method relies on a random walk to reach a statistically valid answer!   Selected Sources:   https://www.osti.gov/servlets/purl/10596 - Igniting the Light Elements   https://library.lanl.gov/cgi-bin/getfile?00326866.pdf - The Beginning of the Monte Carlo Method, Nick Metropolis

Pi is a pretty important number. In theory, it's very simple. It's just the ratio of a circle's circumference compared to its diameter, or C over D if you like. That shakes out to be something around 3. Now, I can already feel the math nerds seething. Pi can't be expressed very easily. It's what's known as an irrational number, meaning it has this unending string of digits after the decimal place.

They don't follow a pattern, they don't make any sense, and they never terminate. So, in reality, it's closer to 3.14 and then an infinite amount of other numbers. As such, we can only ever approximate pi. That is an interesting concept to mull over, an important number that we can never truly know. There are a number of ways to generate this approximation. These are usually pretty heavy-duty calculations. These include integral approximations, series of fractions, spooky trigonometric equations, limits of binomial distributions. The list goes on and on and on. But, and here's the kicker, these are all approximations. At the end of the day, they're all kind of garbage.

We will never know the true value of pi. Thus, at least in my head, my very computer-addled brain, the best method for approximating pi has to be the easiest. Allow me to introduce Buffon's needle problem. The problem itself has some deeper meaning and purpose that I don't really care about right now. It's something to do with the probability of sticks falling across a line. Boring. Nerd stuff. What I care about is that Buffon worked up this equation that can be used to calculate pi, at least indirectly. You can shuffle it around to get pi. So here's how it works. You take a piece of paper and you draw out a series of evenly spaced lines. That spacing has to be the same length as a matchstick or

whatever stick you're going to use in the next step. Now you take a handful of matchsticks and you drop them on your sheet of paper. You count up a handful of matchsticks and you drop them on your sheet of paper. You count up the number of matchsticks and call that total n. Then you count up the number of sticks that are touching a line and call that h. Pi is approximately equal to 2 times n divided by h. That's 2 times the total number of matchsticks divided by the number of sticks that fell on lines. That, dear listener, is the best way to calculate pi. Try it for yourself if you don't believe me. The more sticks you toss, the closer to pi you get. Given enough sticks and enough time, you'll arrive at a fantastic approximation

of pi, all thanks to the power of random sampling. Maybe you can guess where we're going from here. If randomness can be used to approximate pi, then is there some way we can harness this power more generally? Can we turn statistics into a tool? Well, dear listener, we can. Welcome back to Advent of Computing. I'm your host, Sean Haas, and this is episode 116, Haas, and this is episode 116, Monte Carlo. I guess I should really say, welcome back to my algorithmic meandering. So here's the thing. I finally have scans of that Stan Frankel oral history that I mentioned a few episodes ago. I usually don't do these types of follow-up episodes, but I have some good reason for this one. I'm going to start this episode with a quick retread and update before we dive into the main topic at hand.

For those who haven't listened to the LGP30 episode, or who can't perfectly recall a podcast from about a month ago, here's the recap. The LGP30 was a computer designed by Dr. Stan Frankel in the early 1950s. It was remarkably small, simple, and cheap. Due to these attributes, the LGP-30 makes all kinds of weird cameos in the history of computing. The precursor to BASIC, a language called DOPE, was developed on an LGP-30. Another 30 was used for some of the first chaos theory research. It also shows up in this piece of hacker folklore called the story of Mel. Another factor that makes the LGP-30 interesting is this designer, Frankel, was probably one of

the most experienced and well-connected computer scientists of his era. And I'm not exaggerating here. He knew just about everyone in the field. He had worked on just about every computer ever constructed up to that point. And he blazed some wild trails leading up to the LGP-30. For instance, and here's the big excuse for the episode, Frankel was heavily involved with the lead-up to the Monte Carlo method. This was the first time a program was able to exceed what humans could do by hand, or by calculator. Frankel was there while the first Monte Carlo simulations were developed for ENIAC, or at least, he was very close at hand. This was, full stop, the first practical program to ever run on an electronic digital computer. At least, the first practical program that was new and exciting.

So, there is some deep lore here. When I work up an episode, I always run into missing pieces. I like to be on the edge of what's well-researched and well-documented, so this kind of comes with the territory. For that LGP30 episode, I was missing Frankel's voice, for lack of a better term. I was working off a few of his academic publications and second- and third-hand accounts of the man. I went looking for interviews, and all I could find was a physical folder off in the Smithsonian Museum of American History. I put in a duplication request, basically just, I asked for scans, and then I waited. Those scans arrived about a week ago. One big question was left open at the

end of the LGP30 episode. I didn't know what Frankl's intentions were with that computer. Why did he make it? In his academic descriptions, he just says that he had created the most simplified computer possible, but he never says why. This is exactly where I come in with my usual brand of wild speculation. My guess was that he was trying to make something like a proto-personal computer. My only evidence for that guess was the machine's design. It just feels more personal than its contemporaries. Fast forward to today, and, well, we have an explanation in Frankl's own words. As for why he started on the project, he explains in that oral history that it partly came down to boredom. He had been working at Los Alamos and then at various

federal labs up until 1949. That year, he had his security clearance revoked. In his words, quote, the army decided that the country was safer without my help, so we separated. After that separation, Frankel started working at Caltech as something of a computer master, for lack of a better term. He was, after all, one of the foremost experts in the world. His job was to help other researchers get acquainted with the university's new IBM hardware. But, as it turns out, most of the researchers already knew how to use the thing. This was probably for the best, because if you recall from the previous episode, Frankel wasn't really the best with initiates, so to speak.

Long lead-in, but here's how Frankel explains the start of the LGP30 project. Quote, about that time, by which I mean early 1950, it began to seem to me that everybody else in the world was building an electronic computer, so why shouldn't I? So I got thinking along those lines. It was not just thinking, though, because I actually put together a machine using parts that I swiped from wastebaskets at Hughes and such places. Laughter. End quote. Frankel was basically left with a useless job, lots of spare time, and the connections to get lots of spare parts. And thus, the LGP-30 started to form. Now, here's the biggie.

the LGP-30 started to form. Now, here's the biggie. The computer starts partly out of boredom, but there is some ideology behind the machine. Frankel explains in this interview that he was, in fact, trying to make a personal computer. Quote, My aim, perhaps dream is a better word, my vague objective was to see if the computer field could be made more accessible to a person without a big government contract at a small school or a not-massively-financial school like Caltech. So my objective in this design was to pare things down and try to achieve simplicity, if necessary, at the expense of powerful computing ability. End quote.

Let me just underline how wild this is. Frankel, this highly skilled and experienced computer scientist, was thinking about accessible computing all the way back in the 50s. Not only was he considering the idea, he was taking steps towards it. The LGP-30, on its own, provides enough evidence of that. It's really, really fascinating to see that this tiny machine is backed up by such a big dream. For me, that just further cements the LGP-30 as an important part of the canon of computing. Okay, deep breaths now. That's the big update. Where do we go from here? Why, we head back to Los Alamos, the Manhattan Project, and Frankel's early years. You see, there's another reason I wanted to get my hands on this interview. There are two primary researchers that lead us up to the Monte Carlo method,

Stan Frankel and Nick Metropolis. It was, of course, a much larger effort than just two researchers. But these researchers were on the vanguard. They'd bring some of the first programs to ENIAC, and they'd bring ENIAC to Los Alamos. So, here's the deal. In 1987, Metropolis wrote an article for Los Alamos. So, here's the deal. In 1987, Metropolis wrote an article for Los Alamos Science on the history of the Monte Carlo method. In 1972, Frankel discussed his memories of that

time during his oral history interview. I think this puts me in a particularly good position to discuss the first time a computer beat a human. I also think this will be a good companion to our last episode on the early discovery of genetic algorithms. Genetic algorithms in general, and Baracelli's work in specific, show us a really beautiful way to solve complex problems. These algorithms use a model of evolution to approach optimal solutions to problems. It's a really smart and frankly really beautiful way to approach a solution. There's a certain artfulness to it. The Monte Carlo method, on the other hand, is perhaps the biggest brain play in digital history. It's essentially a way

to brute force an approximate solution to a problem. It's the kind of method that's only possible on a computer, since you have to make thousands or millions of guesses to come to an approximate answer. It's crude and it's coarse, and that lends it a unique type of beauty. So this episode, we're going to be looking at the very earliest roots of this method. We're going to see how Frankel and Metropolis both contributed to its development and look at the final leap that took us beyond what a human can normally do. New Mexico is called the Land of Enchantment for a good reason. It's one of those parts of the world that feels uniquely magical. I've traveled there a handful of times myself, and it always

lives up to expectations. The sky almost feels closer to the ground in that land. I don't have a better way to describe it. Maybe it's just something you have to feel for yourself. don't have a better way to describe it. Maybe it's just something you have to feel for yourself. Situated near the middle of the state, just north of Santa Fe, is Los Alamos. The lab itself was constructed on top of a mesa, a type of flat-topped hill that dots the New Mexican landscape. In that way, I guess Los Alamos was made even closer to the sky than the surrounding lands. Perhaps this was the only place that a lab like Los Alamos was made even closer to the sky than the surrounding lands. Perhaps this was the only place that a lab like Los Alamos could exist. It holds a weird position in history in general. For my purposes, it's always held this duplicitous role in the development of computing.

Frankl himself puts it bluntly, quote, The function of Los Alamos was to put genocide on a paying basis. The bombs were designed, developed, and experimented with and built in Los Alamos. The research done at Los Alamos was directly responsible for the deaths of hundreds of thousands of people. The very existence of nuclear arms has led to decades of conflict the world over. There are certain frightening realities of the products of Los Alamos. This lab was a catalyst for war and destruction. It was also a catalyst for massive technological advances. That is, in part, the result of concentrating highly

talented and driven scientists on an isolated mesa. One such development was artificial nuclear fusion. That isn't a new phenomenon, far from it, in fact. Fusion's been going on ever since the very beginning of the universe. It was already understood, at least in part, as early as the 1920s. By the 1930s, fusion was being carried out in the lab, or at least small-scale fusion tests were being conducted inside very fancy particle accelerators. Before I go any further, I should warn you, we need to get into the realm of physics to appreciate today's story. Fusion is what powers stars. It's the reason we all exist here in the first place. The universe doesn't really come together without this phenomenon.

That said, it's deceptively complicated. complicated. The short version is that fusion is a process by which two atoms smash together, which forms a new atom and releases excess energy. If that was the full truth of the matter, things would be a lot simpler. In reality, fusion only occurs under very extreme and specific circumstances. It all comes down to the interplay between the Coulomb force and the nuclear force. An atom is made up of three smaller particles, electrons, neutrons, and protons. Today, we actually get to disregard the electrons and focus on the two bigger particles. Neutrons, as the name suggests, have a neutral electrical charge. Protons are positively charged.

I'm sure we all can guess the implication here. Protons don't like to get near each other. It's like trying to push together the positive ends of two magnets. You get this repulsive force called the Coulomb force. This is a long-range force. Technically, it's a force that can be felt at any distance, but in practice, that's just a math thing. The strength of the Coulomb force goes down by radius squared, so once you get a little ways away, the force isn't very powerful at all. If this were the only force at play, then we wouldn't get very complex atoms.

At most, we'd have atoms composed of one proton, a neutron, and an electron, aka hydrogen. And not even exotic hydrogen, just the normal boring hydrogen. That would be a pretty boring universe. Luckily, there are more forces at play that make for more interesting chemistry. Protons and neutrons are actually attracted to each other via the nuclear force, aka the strong nuclear force. That's what holds atomic nuclei together. But there's a catch. The nuclear force is a very short-range phenomenon. We're talking 10 to the minus 15 meter range. Subatomic particles have to almost be touching for the nuclear force to take effect. But once it does, it's more powerful than the repulsive effects of

the Coulomb force. This allows for more than one proton to be smashed together, which in turn gives us elements heavier than hydrogen. This sets up something interesting. Hydrogen is the only element that will form on its own if you just have some spare subatomic particles around. Once you get hydrogen, you run into a problem. The Coulomb force should, in theory, prevent other protons from entering a hydrogen nucleus. To get cooler elements, you need a way to overcome that force. Us humans first attempted this using a particle accelerator, basically smashing atoms together with enough energy to

get them to smush and touch. Classically speaking, this shouldn't work. Or, it should work, but very, very occasionally. The Coulomb force is strong enough that it can't be overcome without really absurd amounts of energy. Under the laws of classical physics, the atoms should just bounce off each other. The charge on each proton should be enough to ensure that. But there's another aspect at play. You see, fusion is possible in large part thanks to quantum tunneling. This is actually a really, really complicated phenomenon. When we say quantum tunneling, we don't mean that some particle tunnels through a physical barrier. We're instead talking about potential barriers, as in energy. Think a force field. The Coulomb force

creates this potential barrier around the nucleus that should take a pretty large amount of energy to overcome. Think back to magnets. You can make two positive ends of two magnets to touch, but it takes a lot of force. When you smack those magnets together, you're overcoming a potential barrier. To get over those potential barriers, you have to put in energy. That can come in the form of high-speed collisions, a whole lot of heat, a whole lot of pressure, maybe in some cases a type of electrical charge, but the point is you need an absurd amount of energy. But once you

overcome that barrier, you can push together your nuclei and allow nuclear force to take over, which leads to fusion. That said, we're talking a lot of energy. If you do the math, which some smarter folk than I have done, it turns out that fusion should be very uncommon. But we do know, however, this isn't the case. If you want evidence, then look around you. We have a lot more than hydrogen. We even have things like carbon and iron. The sun

even burns pretty hot. You know, all that good stuff we get as a result of nuclear fusion. The secret here, the part that solves this problem, is quantum tunneling. This is a phenomenon in which small particles can actually tunnel through a potential energy barrier. So instead of building up lots of energy to overcome the Coulomb barrier, some particles actually just kind of pop through it. Think about trying to push two magnets together and all of a sudden, they just slam. No energy put in, just all of a sudden you're through that potential. The specifics here get even more complicated, so I'm not going to get into it. If you want to know more, please, I implore you to go take a class in quantum mechanics.

I haven't taken one in years, so this is about where my knowledge ends. I hope you like calculus if you choose that route. What matters is that tunneling is a probabilistic phenomenon. It doesn't happen all the time, it just happens on occasion. You can increase the odds of tunneling by keeping lots of hydrogen atoms close together for a long period of time. That's exactly how stars beat the house, so to speak. Inside a star, hydrogen atoms are smushed together under the force of gravity. Huge amounts of gas are bound up into these big stellar balls. That creates an unimaginable pressure and temperature at the heart of a star.

You have a lot of hydrogen atoms in contact for a long time, and you have a lot of potential energy. In other words, a lot of fusion happens, either by breaking the Coulomb barrier normally, or by tunneling. There's one more piece of the physics pie that I need to introduce here. Have you ever heard of this guy named Albert Einstein, I think it's pronounced? Among his contributions to the world of physics is perhaps the best-known equation in the world, the mass-energy equivalence. That's right. This is one of the few cases where E equals mc squared actually matters. This equation relates mass to energy. In other words, it says how much energy is bound up in an amount of matter.

It works best on small scales. And we're in a pretty small scale when we're talking about fusion. This equation allows for a fun type of accounting where researchers can sum up the energy present in reactions. Some reactions require outside energy to pop off. Others produce their own. Fusion, it turns out, falls into the second camp. Fusing two atoms as long as they are relatively small will lead to an excess of energy. This is, of course, assuming the Coulomb barrier has already been broken or tunneling has already been achieved. We're just talking about the reaction between two nuclei. If handled properly, this could lead to a feedback situation.

You fuse two hydrogen atoms, usually actually tritium and deuterium atoms for math reasons, and you get a helium atom plus an extra neutron and a burst of energy. That energy is released as heat. That heat can add extra energy to the system, which can help fuse other atoms, which in turn provide more energy. You get a chain reaction, at least in theory. This type of chain reaction enables stars to ignite and continue burning for eons. It could lead to near-limitless energy, but it could just as easily lead to near-limitless destruction. The main goal of the Manhattan Project was to create a fission bomb.

Now, fission only actually slightly differs from fusion. In fission, an atomic nucleus is cleaved into multiple smaller pieces, which in turn releases a burst of energy and some extra neutrons. Those free neutrons, now speeding around, go on to cause other fission events. So, in a way, we're kind of looking at the opposite of fusion. Fission, however, it turns out is a lot easier to kickstart. To get a runaway reaction, you only need to create a critical mass of some fissile material, like U-236 or plutonium.

Once that criticality is reached, a chain reaction starts to occur somewhat naturally, which can then be pushed towards an explosion. The math for modeling this reaction is difficult, but it's not impossible to do by hand. At Los Alamos, teams of human computers were doing just that. They were assisted, of course, by desk calculators. Fusion enters the picture in an almost offhanded way. Apparently, it started as something of an idle conversation. The story goes that Enrico Fermi and Edward Teller, two physicists on the Manhattan Project, were taking a stroll in 1941. This was at the very beginning of the project. One can imagine that the cement foundations of some of Los Alamos' labs were still drying. Supposedly on this walk, Fermi started wondering

aloud if a fission bomb could be used to initiate a fusion reaction. Teller couldn't get that idea out of his head. Somehow it just struck like the perfect nerve inside his brain. He would spend the following years driving towards that idea, a fission bomb igniting a fusion reaction. In a way, the creation of an artificial star. The qualitative part here is simple. Fusion, of course, requires a huge amount of energy. But once you put that power in, in theory, you get more power out. A fission explosion should bring enough heat, pressure, and perhaps radiation to the equation. Fission was, at least in part, a solved problem. That is, it was a known phenomenon that had been demonstrated in a lab.

Some of the finer points were a little fuzzy, hence the army of number crunchers, but there was a firm theoretical grounding prior to the Manhattan Project. Fusion, on the other hand, represented a bit of an unknown quantity. The physics was there, at least in theory. The big question was whether or not a chain reaction was feasible. And if so, what conditions were needed to start a chain reaction? This was basically the do or die question that the entire project revolved around. A fusion bomb was only possible if a chain reaction could be sustained. The thing would only blow up if one fusion event could

release enough energy to set off another fusion event. In fact, you probably need more than just one event to follow the first. These unanswered questions seemed to only spur Teller on. Apparently, he saw fission as somewhat boring. In truth, it was almost a solved problem. In truth, it was almost a solved problem. There was a path forward. That wasn't so much the case with Fusion. Teller became so engrossed with this fusion bomb, which would be codenamed the Super, that he would actually neglect his other duties.

Instead of contributing to the fission bomb, you know, the Manhattan Project's main goal, he would focus on the Super. This is where we reach one of the many brick walls on the way to fusion. There wasn't a way to test fusion in the lab, at least not to test chain reactions in any large-scale manner. It was possible to fuse elements in atom smashers, but that was only a single event, maybe one or two subsidiary events after that. But that was only a single event, maybe one or two subsidiary events after that. That didn't really help in the study of chain reactions. Fission, on the other hand, was very easy to study in controlled conditions.

The first nuclear reactor, the Chicago Pile 1, had sparked to life in 1943. It and similar reactors allowed scientists to observe controlled fission chain reactions. You could get readings and run honest-to-goodness experiments. This was one of the ways that fission became a solved problem. For fusion, you had nothing. There had been at least one attempt at a fusion reactor, but it was met with failure. The only option was to resort to mathematics, which came with its own world of headaches. The math to simulate reactions was simply too complex. It would take too much time to run the numbers, even with fancy IBM punch card equipment. This is where the usual

connections come into play. John von Neumann enters the picture at this point, and I have actual evidence that we should really call him Johnny. Frankel states multiple times that he never went by John in person, always Johnny. Anyway, Johnny von Neumann was a pretty well-connected dude. He consulted for the Aberdeen Proving Grounds in Maryland, and he was also a regular at Los Alamos. Why does this bridge matter? Well, it all comes down to artillery. Back in the day, artillery had to be set up using firing tables. These were field manuals that contained tables for conditions and target ranges. You'd look up your current numbers, and the table would tell you how to set up your specific gun. current numbers, and the table would tell you how to set up your specific gun. In reality,

these tables were the product of a set of differential equations. They had to be calculated by hand for each gun and each type of round in use. With the rapid pace of development, more and more new tables were needed all the time. So Aberdeen went out searching for a way to replace their human number crunchers. The solution they landed on was a project called ENIAC. Through a series of connections and bids, Aberdeen ended up funding the construction of this computer at the University of Pennsylvania. That machine, the ENIAC, was built in secret. It was to be something of a secret weapon, eventually finding its way down to Aberdeen to calculate firing tables full-time. Good old Johnny von Neumann was one of the few people in the world

that actually knew about ENIAC. He had seen the machine under construction, so it's no surprise that he was looking for a more interesting use of the computer. Firing tables are fine, sure, but a mainframe needs more. Teller and von Neumann were well acquainted. Perhaps they even went on walks around Los Alamos. One thing led to another, and Johnny would find out about Teller's fusion issues. By late 1945, INIAC was nearing completion, and von Neumann sprang into action. Nick Metropolis explains it this way in The Beginning of the Monte Carlo Method. Quote, he wondered whether Stan Franklin and I would be interested in preparing a preliminary computational model of a thermonuclear reaction for the ENIAC. He felt he

could convince the authorities at Aberdeen that our problem could provide a more exhaustive test of the computer than mere firing table computations. End quote. Frankel and Metropolis, two physicists working with Teller, really jumped at the idea. Of course, they had no clue what this computer thing was, but the idea of automating away their math woes, well, that must have been exciting. In short order, the three, Frankel, Metropolis, and von Neumann, were off to Pennsylvania to put the ENIAC through its paces. At least, that was the initial plan. It turns out that this idea was a little too complicated. Frankel and Metropolis wanted to do a full simulation of fusion. I guess they didn't really have a handle on what the ENIAC was capable of. Or,

rather, no one really did, outside of one lab in Pennsylvania. As Frankl put it, quote, that problem was ill-advised. It was vastly beyond the abilities of ENIAC. What we hoped to do was a quite unsuitable attempt, end quote. This is one of those scenes that I just love to imagine. Von Neumann swears he has a great problem that will put the ENIAC through its paces. It's going to be the first honest-to-goodness program the machine runs. It will be a much better test than simple firing tables. Two intrepid young physicists arrive to deliver a problem, and, well, the problem simply can't run on any hack. I imagine there is a great collective sigh as Franklin Metropolis sketched out a big

equation and everyone in the room realized it wasn't going to work out. Now, we don't know exactly what this initial problem was. From what I've read, I assume this first pass was a fully quantitative model, as in, a model that calculates every little thing using actual numbers and real equations. Back in my day, I wrote some software to run these types of models. You basically start at the particle level, figure out how many particles you're going to simulate, populate a 3D space with your simulated particles. Then you start running equations.

You make little time steps where the state of the model is updated by fancy and very precise physics equations. I've personally written this kind of software for modeling gravitational interactions, but the overall structure should be similar for fusion. Why would ENIAC have an issue with these types of simulations? Well, let me count the ways. ENIAC didn't really have random access memory. Its only real storage, or at least real rewritable storage, was made up of 20 registers, each able to hold up to 10 decimal digits. You can also configure constants using a big bank of dials. In other words, you don't have actual memory. You have this weird type of almost RAM

and this weird type of almost ROM. These qualitative models require a lot of memory space. You need to track information about every particle. So let's say we're working up a fusion model. Each particle has a location in 3D space, so we need a variable for x, y, and z coordinates. We also need to track the particle's type. Is the particle a neutron or a proton? We need some way to track velocities, so add vx, vy, and vz variables, since it can have velocity in three different dimensions. There are also some trade-offs that we'll need to make. If we want to make the code run faster, we could store energy and a few other parameters in memory, or to save space, we could calculate those points on the fly, if we're willing to have the simulation run a little

more slowly. The bottom line is ENIAC didn't have enough space for all of this. Maybe you could store a handful of particles. Maybe you could pull some tricks. But that would ruin the simulation. This type of model works by dealing with large amounts of particles. You have to have space. Another problem comes down to accuracy. Frankel admits that to get software running on ENIAC, you had to compromise on accuracy.

Instead of calculating numbers out to the 100th decimal place, you had to settle for 5, or maybe even 1. That means that each calculation includes some type of rounding, which introduces an amount of error. Pi is only 3.14 instead of 3.141592683 and so on. 1 over 3 is treated as 0.33 instead of 0.33333 and so on. These are small differences on their own, but if you're running a lot of calculations, those differences add up quickly, so over time, you get more and more error. You get less and less accurate. Despite these issues, the trip out to ENIAC was well worth the time. Frankel and Metropolis became some of the first people to see the ENIAC in action. They were probably the first outside of the small lab to actually see

the machine running. They both describe getting a tour of ENIAC and learning the fundamentals from its designers and programmers. Here's where we hit a bit of an awkward bridge in the story, though. In April 1946, a meeting is held about the future of the fusion bomb. There's this paper presented at the meeting titled A Prima Facie Proof of the Feasibility of the Super. This, apparently, claims that current research shows a fusion bomb may be possible. The result of this meeting would ensure continued interest in a fusion bomb, at least for the time being. The paper itself is still classified.

At least, I definitely can't get access to it. What we do know is there is some program that was run called the Los Alamos program that actually could work on ENIAC and contributed to this 1946 paper about the super's feasibility. So let me pull out some of the facts and see where we can go with this, see if we can understand what the Los Alamos problem was. We know that the ENIAC played some role in this. Herman Goldstein, one of the researchers in the ENIAC lab, explained this period in his book, The Computer from Pascal to von Neumann. It's a bit of a long passage, so I'm just going to paraphrase. Goldstein didn't have clearance to know about what was going on at Los Alamos. He just knew that his buddy Johnny was bringing in

some folk to run the first program on ENIAC. These two researchers, Franklin and Metropolis, came armed with a set of equations. They worked closely with Goldstein as well as Mouchley, Eckert, and the original team of six ENIAC programmers. Some type of program was developed, which we can guess had something to do with the super. A letter was later sent to Goldstein thanking him for his contribution to some secret project at Los Alamos. Now, this actually complicates matters for me somewhat. Goldstein doesn't mention a first failed attempt. So the Los Alamos problem must have been an actual functional program, and I'm guessing it was the second attempt, the second pass at a program. We have to take some more creative liberties to try and figure out what that program was. Franklin Metropolis don't offer very many clues,

probably because of security clearance things. But one interesting idea comes out of Dyson's book, Turing's Cathedral. He paints a picture of a big calculation occurring in late 1945, leading into early 1946. This calculation was the true shakeout of ENIAC's circuits. Punch cards were used as a way of getting around ENIAC's small memory space. Input data was punched onto cards and read into the machine as needed. Intermediate values were punched onto cards for later steps, thus freeing up very scarce register space. Final results were also, and get this, punched onto cards. Final results were also, and get this, punched onto cards.

Dyson cites to information from the ENIAC team and trial testimony. There was this whole patent dispute in the 70s called Honeywell vs. Sperry that had to do with the origins of the electronic digital computer. Part of that trial comes down to when ENIAC was actually operational. Now, it should be said, I have some issues with these types of sources. I don't think we should take testimony from a patent case as gospel. All parties have a vested interest in their testimony being a certain way. So, there is an angle here. Likewise, we need to be careful with sources from the ENIAC team. They didn't have security clearance, so they knew the methods, but not the reasons that those methods were being used. The best information I have comes from a paper titled Igniting the Light Elements,

the Los Alamos Thermonuclear Weapon. It's another publication from Los Alamos themselves. The paper gives us a pretty loose description of the Los Alamos themselves. The paper gives us a pretty loose description of the Los Alamos problem. It was apparently a set of three differential equations, that's three big gross calculus equations. The goal was to simply determine if a fusion reaction would propagate and cause a chain reaction. In order for this program to work, the model, a very quantitative one, had to be heavily simplified. It was taken from three dimensions down to one, forcing a small number of particles to live on a single line. The numbers were truncated down to fewer decimal places.

Frankel even explains this trick, where the ENIAC registers could be broken up into smaller chunks of digits. So the actual program may have been using up to 40 registers, each 5 decimal digits wide. This problem was still pretty complicated, but at least it was possible to execute. The actual calculations would take around 6 weeks to complete. In the end, it was shown, at least assumedly, that a fusion chain reaction was possible. Now, we still don't have a super good idea of the comp methods at use in the Los Alamos problem, but I think we can make an observation here. At this point, we're still in the realm of glorified calculators. The Los Alamos program made pretty full use of ENIAC's computing power, but it didn't do anything computationally wild.

By that, I mean the problem was solved by simply using fast mathematics. It could have been done by hand if given enough time and power. The whole point in using ENIAC here was to speed up the process. hour. The whole point in using ENIAC here was to speed up the process. In fact, I'd wager the final program used many of the same methods employed by human calculators over on the Mesa. You see, ENIAC was a weird computer. I hope that I've made that much very clear. Many aspects of its design were borrowed from earlier mechanical calculators. The great leap forward was to electrify and automate everything. This means that a lot of the mathematical methods used on old hand-cranked calculators, and even on some fancy IBM punch card machines,

would have also worked on ENIAC. We're still in this regime where progress is happening by stepwise improvements. At least, this is the impression that I've gotten from the context around the Los Alamos problem. Now, this is the point where most histories jump right to the Monte Carlo method. But that glosses over one final stepwise improvement. So, here's the rub. Stan Frankel is about to leave the picture. He was instrumental in the prehistory of the Monte Carlo method, but the final leap occurs without him. That said, there is an interesting contribution here that often goes unmentioned. In July 1946, ENIAC is disassembled

and moved from the University of Pennsylvania to its final home at the Aberdeen Proving Grounds. While there, the machine will be upgraded, improved, and enter into constant use. Shortly after that move, in 1947, Frankel and Metropolis publish a paper titled Calculations in the Liquid Drop Model of Fission. The physics here is, frankly, over my head. From what I understand, the liquid drop model is an approximation of nuclear fission. What we're looking at is another simplified model, like the Los Alamos problem. It would provide some kind of hand-wavy results. But here's how Frankel describes that research during his oral history interview. Here's how Frankl describes that research during his oral history interview.

Quote, It was a calculation repeated many times for many choices of parameters. Essentially, we were calculating the energy of a deformed nucleus, and the shape of the nucleus was specified by about four or five parameters. We would choose values for these parameters, set them into the machine, I guess with a hand-turned crank or dials, and then tell the machine, okay, go ahead and do it. I think it calculated for something like 30 seconds, and then it would punch a card with the results. Now, to do the same thing with an electric desk calculator would have been some hours of work. Since we cranked out perhaps

a few thousand, it would have been beyond human ability to do it by hand. The model would have been loaded up into ENIAC. At this point, that had to be done by wiring up plug boards. Constants would be configured using rows of dials. For the liquid drop program, those constants would be treated as parameters for equations. By fiddling with these dials, Frankel and Metropolis were able to quickly and easily run their model for different conditions. They were able to sample different possible inputs. That might sound mundane at first, but this is actually a pretty big step. When you're working with a model, you often want to optimize it for some set of conditions. Take fusion as an example.

Maybe you want to figure out the optimal starting density for a certain fuel. Sometimes you can solve that analytically by reorganizing equations. But other times, you can't. Maybe the math is too hard. Maybe there's some ambiguity in the equations. Maybe the model doesn't use reasonable Maybe there's some ambiguity in the equations. Maybe the model doesn't use reasonable equations or any equations at all. One option is guess and check. You pick some parameters that you think are optimal. You plug in those values, then you calculate your

results. Then you do it again and again and again and again. For a guess and check to work, you need a whole pile of results so you can spot trends and prove that some guess is the best solution. That takes a lot of time. In this liquid drop study, Frankel and Metropolis are using ENIAC to automate this checking process. All they have to do is guess, then ENIAC does the check for them. This, in my humble opinion, is one of the small seeds that leads to something wonderful. The next leap happens roughly concurrently with this liquid drop research. We're talking late 46 into spring of 47. And this isn't just going to be another step. This is an honest-to-goodness jump.

It came courtesy of one Stan Ulam, yet another high-octane researcher at Los Alamos. As Metropolis recalls, he and Frankel had presented an overview of ENIAC a year prior. Something about the computer intrigued Ulam. You see, Ulam was a bit of a statistics nerd. That interest would help push the team towards the Monte Carlo method. From Metropolis, quote, The spirit of this method was consistent with Stan's interest in random processes. From the simple to the sublime.

He relaxed playing solitaire. He was stimulated by playing poker. He would cite the times he drove into a filled parking lot at the same moment someone was accommodatingly leaving. Beyond just games, Ulam was fascinated by the idea of multiplicative or branching processes. Metropolis makes special mention of this, so I think I should do the same. These are the formalization of chain reactions. In a multiplicative process, one event can trigger multiple events on down the line.

Those second-generation events can, in turn, trigger a third generation of events, and so on. During his time at Los Alamos, Ulam had conducted extensive research on these types of reactions, going so far as to model them statistically. Initially, Ulam was interested in applying this new ENIAC thing to the fission research conducted at Los Alamos. This was what he knew best, after all. One of the outstanding problems that had proven hard to calculate by hand was neutron diffusion. When a fission event occurs, a heavy element is broken into a number of lighter elements,

plus a number of fast-moving neutrons. Those neutrons can then break apart other heavy elements, which, in turn, creates more neutrons. Thus, a chain reaction is set up. So these neutrons are of prime importance. Specifically, you want to know what happens to a neutron after it escapes its nuclear bonds. Does it fly off into space? Does it strike another atom? Does that strike lead to more fission?

Or does the neutron just bounce off? And the biggie, what kind of conditions are best to ensure a chain reaction? In other words, we're dealing with an optimization problem. This would seem to be impossible to solve given the complexity of the mathematics. But Ulam had a spark of an idea. You see, the actual numbers don't matter all that much. The point was to work out starting conditions. The model itself didn't have to be fully quantitative.

It could be an approximation. In fact, it could be a pretty bad approximation and still work out. Ulam believed that he could model the entire thing statistically. That is, if he had access to a computer. And so the Monte Carlo method took shape. Ulam and Metropolis were the two primary researchers to develop this method on a computer. Metropolis himself would actually name the method after the Monte Carlo casino. The story goes that this was chosen because Ulam was always talking about an uncle that

would lose grand sums of cash at the tables of that casino. So, what does the method actually entail? This is perhaps the best part. The Monte Carlo method uses random numbers and repeated calculations to approximate a solution. The fission case here, the first Monte Carlo simulation, is a wonderful example to work off of. The simulation starts out with a single neutron hurtling through space. It has some energy level to start with. The simulation then calculates how far the neutron can travel before it hits an atom.

Once it hits that simulated atom, the gears really start to turn. The program decides what should happen to this atom. Will fission occur? Will the neutron simply bounce off? Well, that's determined by a number of factors, but by and large, it's steered by random chance. The program has some function that can approximate the probability of fission occurring. It might even take the neutron's energy into account, or it might just throw some dice. If fission occurring. It might even take the neutron's energy into account, or it might just throw some dice. If fission occurs, then the computer starts tracking the resulting neutrons, and the same calculation happens all over again. You wait for a hit to happen, you roll some dice,

maybe look at the neutron's energy, and then calculate the next step. Metropolis describes the Monte Carlo method as generating a lineage of neutrons. This program ends up spitting out a family tree showing how many descendants a single fast neutron created. Here's what's so cool about this method. At no point are any heavy-duty calculations needed. You take a step, randomly decide the outcome of the step, then you take the next step. Repeat ad nauseum and you eventually arrive at a conclusion. All the real math is done beforehand when you calculate probabilities for events. In this simplified model I've presented, you only really need one type of event, fission, and that's simple fission at that. You can add in

more options to make a more realistic simulation, but everything stays relatively simple. Step, roll, step, roll, step, roll. Usually these models will run for some set number of generations. Then you calculate some resultant value. For this fission model, it could be the number of neutrons or some associated reaction energy. Once that's done, you store your final values. Then you run the whole thing again and again and again. You want to run the simulation as many times as possible. Remember, it's all a statistical model. Your final results will change between runs, so you need repetition. Eventually, you'll get enough data to apply statistics. You take an average of the final energy, you make a nice probability distribution, and you're good to go.

You now have a pretty robust analysis of your model. You have a pretty good way to describe the phenomenon. The Monte Carlo method, despite being so simple, works really well. It's also something that humans can't do by hand. You need a computer or some type of machine. In fact, it relies on the automated nature of a computer. Millions upon millions of calculations have to be carried out. Each is small, simple, and repetitive. To do this by hand would take years of someone's life, perhaps longer than a human lifespan. It would also be susceptible to operator error at every step. In the digital realm, well, this is a cakewalk. It's a method that's only viable on a computer. It's almost purpose-built to be run on a digital machine.

I think it's also interesting to look at the Monte Carlo method in comparison to another optimization technique, the genetic algorithm. I did tell you last episode, I've been in an algorithm mood lately. Now, think back to that episode, only a few weeks ago. Genetic algorithms use the framework of evolution to solve a problem. You are, in effect, creating a lineage of digital organisms purpose-built to solve your problem. Over many generations, you arrive at an optimal solution, backed up by the science of biology. It's elegant, it sounds smart, and it's just cool.

The Monte Carlo method is much more brute force, but I think it's in a cool way. I like to think of genetic algorithms as these little terrariums filled with critters. The Monte Carlo method is more like a working lab. Through random sampling, you try out every possible option, or at least you try out enough options to have a suitably large dataset. You're randomly sampling all possible outcomes. Once you finally get all your results, you're left with a picture of all these outcomes. You can use that to make better guesses about the process, refine theories, predict outcomes, and even cherry-pick an optimal solution.

At this point, it may sound like we're at a comfortable conclusion, but there's still a wrinkle that I want to work out. How does this all tie back to fusion? So far, Ulam and Metropolis are only using the Monte Carlo method on fission simulation. That's still a massive leap forward. This is still, as far as I can tell, one of the first practical computer programs that outpaced human-powered mathematics. This is the first time that ENIAC is used as more than a souped-up calculator. But how does this all come back to fusion?

At the end of World War II, Los Alamos was left in a weird spot. The US had cornered the market on mass destruction. Some saw the atomic bomb as the ultimate weapon of war, and conversely, the ultimate weapon of peace. If America was the only nation with the atom bomb, then how could there ever be another war? How could there ever be another aggressor? Thus, the very idea of a bigger bomb was kind of sidelined. There was a bit of a problem with that idea, though. The whole notion hinged on the US being the only nation with a bomb. That wouldn't last very long. In fact, Teller personally had a hand in this. During the war, Edward Teller had neglected most of his work on the fission project. The super was

just more important and interesting to him, after all. In order to pick up the slack, his fusion work was pushed off to other researchers, which included Klaus Fuchs, who turned out to be a spy working for the Soviet Union. This would only be one contributing factor, though Salomos seems to have leaked kind of like a sieve in this period. Regardless, it also wasn't impossible for another nation just to make a bomb on their own. It just took time, money, and physicists. The Soviet Union tested their first atomic bomb, RDS-1, in 1949. That was enough to get funding pouring back into the superproject. Researchers in Los Alamos had never really given up on the idea of a fusion bomb, they had just been slowly grinding away at the problem ever since Teller's obsession

began. The added federal vigor pushed that research to the forefront again. One of the researchers that was still on the fusion train was Johnny von Neumann. Nearly a year before RDS-1 ignited in 1948, von Neumann had been looking at a new idea for a simulation. He and his colleagues figured that it should be possible to adapt ENIAC's Monte Carlo program for fusion. This is another wonderful thing about the method, or scary thing about the method depending on its results. The same framework can be used to model just about anything. The model can become more complex, used to model just about anything. The model can become more complex, the model can become bigger, but it'll still work. It's still relatively easy to adapt. It would take a number of years and

countless trials and errors, but the simulation route eventually bore fruit. And thus, the Monte Carlo method, a technique inspired by fusion research and first used for fission calculations, found its way back to the super. Alright, we've come to an ending of sorts. The Monte Carlo method is interesting to me for a number of reasons. It's one of those computational methods that, on the surface, kind of sounds absurd. It honestly sounds more like a game than a modeling method. You just do a random walk around a simplified idea of physics, and if you take enough steps,

you get a valid answer. It's a solution that's almost elegant in its ham-fistedness. The history of the method is just as improbable. It's outlined by these furtive steps that eventually lead to a final jump. Now, many histories just go straight from the Los Alamos problem to the casino, but I think that's rushed. I think we have to at least nod at Frankel and Metropolis's liquid drop model research. That program used a very manual form of parameter sweeping. Frankel describes physically turning dials to try different parameters, and there's

something I love about that. Once enough parameters were tested, they could build up a statistical view of the model. They could find the answers they wanted. That sounds a lot like the Monte Carlo method, just far less automated. Throw in some random numbers, maybe just skip the math in favor of a probability distribution, and you're already at the casino. So where do we go from here? Well, I'll leave you with a fun challenge. If the Monte Carlo method sounds at all interesting to you, then I highly suggest you try writing your own simulation. The actual programming for the method can be remarkably simple. Write up something easy, then keep adding in complexities. It's a fun exercise, and who knows, maybe you'll land on

something interesting with your results. Thanks for listening to Advent of Computing. I'll be back in two weeks with another piece of computing's past. And hey, if you like the show, there are now a few ways you can support it. If you know someone else who'd be interested in the history of computing, then please take a moment to share the show with them. You can also rate and review the podcast on Apple Podcasts. If you want to be a super fan, then you can support the show directly through Advent of Computing merch or signing up as a patron on Patreon.

Patrons get early access to episodes, polls for the direction of the show, and bonus content. You can find links to everything on my website, adventofcomputing.com. If you have any comments or suggestions for a future episode, then go ahead and shoot me a tweet. I'm at Advent of Comp on Twitter. And as always, have a great rest of your day.