99% Invisible - 392- The Weather Machine

Episode Date: March 4, 2020

The weather can be a simple word or loaded with meaning depending on the context -- a humdrum subject of everyday small talk or a stark climactic reality full of existential associations with serious disasters. In his book The Weather Machine, author Andrew Blum discusses these extremes and much in between, taking readers back in time to early weather-predicting aspirations and forward with speculation about the future of forecasting, including potentially dark clouds on the horizon. The Weather Machine

This is 99% invisible. I'm Roman Mars. Andrew Blum is a journalist who writes about some of the biggest infrastructure projects in the world. His specialty is revealing how systems we think of as intangible like the internet are actually made up of very real stuff. The internet relies on cables and wires and data centers, which are maintained by actual people who keep the whole thing running. A few years back, Andrew got interested in the weather forecast.

It's this mundane everyday service that, like the internet, is made possible by a vast and interconnected global machine that took decades to build. The system is a huge scientific project. But it's also a diplomatic one. The atmosphere crosses all political boundaries, and so knowing the weather requires international collaboration. As weather becomes more extreme, the forecast becomes increasingly important, but ironically, because of its growing value, there are now forces threatening to undermine the global system that makes it possible.

It is fascinating stuff. I talked to Andrew about his book, The Weather Machine, and he told me that he first got interested in the forecast back in 2012. It was a kind of busy season for me. My first book had come out. My dog had died. My son was bored. It all kind of happened at once. And there was a weekend afternoon as kind of Sunday in October when I had my kind of new board in one hand and my phone and the other. And there was a weekend afternoon, a kind of Sunday in October, when I had my kind of newborn in one hand, and my phone and the other. And I was on Twitter. And all of a sudden, the meteorologists who I followed kind of went into a tizzy. They all just kind of erupted all at once, based on the output of a weather model. And what they were seeing

was a storm that they had kind of been watching out in the Atlantic, in the Southern Atlantic, but suddenly it was going to turn left towards New York where I live. And it was remarkable because this was eight days ahead. This was a big storm potentially. And they all kind of trusted the output of this model. They weren't, you know, they weren't saying this is definitely going to happen, but it was so far ahead of sort of what I understood as the work of meteorologist, especially kind of hurricane forecasters.

Yeah. And this storm that you could kind of see eight days out eventually became hurricane sand. It is chaos along the Jersey shore. The super storm already stretching across one third of this country from Florida to Canada. I can't wait to see news. I covered New York weather for 25 years. I have never seen water in Lower Manhattan.

There is water now on the streets in Lower Manhattan. I mean, the overall feeling when the storm actually came was that our kind of luck had run out. That New York city had sort of finally begun to reckon with what the storms of the future might be like. With the subways flooded and shut down, nobody did anything for that week. Along the coast, it was months and years, and if you live on the L train, it's still being fixed.

These are the consequences if it was really clear. I mean, 147 people were killed. But when it came, for me, it was a recognition that that forecast eight days ago was right. That an eight day forecast is not stuff of science fiction, but had just happened in the most consequential way. And the real difference here between the idea of knowing a thing that's coming in the way that knowing a cold front is coming or knowing that a tornado is coming is like Hurricane Sandy didn't exist eight days before. It was just particles in the atmosphere moving around and it was a mathematical model that predicted that it would form into this thing that would affect people so dramatically.

I mean, the experience of Sandy made me want to know not only what the weather models were, but where they came from, sort of who built them, how they had evolved over time. I mean, I recognize them as this kind of complex global infrastructure, but as is often the case with complex global infrastructures, their authorship was really vague and longstanding. Right. And so, you know, you've spent a lot of time

thinking about physical infrastructure, and there's something about the weather forecast that kind of has this vibe of face-about mathematical models and physics and stuff in the air, but it still really is rooted in infrastructure. It kind of dovetailed with the thing that you already care about. So like, what is the modern weather machine actually physically look like? Well, to kind of see it like that, you kind of have to have this hallucination about, you know, a sort of planetary scale. It's made up of so many kind of tens of thousands of tiny little pieces.

I always like when you're flying out of LaGuardia Airport in New York, if you're lucky, you kind of pass by the weather station there by the runway. And it looks like a kind of jumble of equipment. And that's one piece of the weather machine. When we see satellite pictures, the kind of jumble of equipment, and that's one piece of the weather machine. You know, when we see satellite pictures, you know, the kind of familiar weather satellites, it's kind of another piece of the weather machine. And then that's repeated, you know,

tens of thousands of times all over the world. Yeah. So as you started looking at the history of the weather forecast, now it started. You found that it was actually a revolution in telecommunications that made the first weather forecast possible. So tell me why the development of the telegraph was important for understanding the weather.

It's really about having this picture of the Earth across space. We have maps, that's kind of one way of imagining the Earth, but until you can communicate instantaneously across distance, you know, basically until you can communicate instantaneously across distance, basically until you have the telegraph and then all of the communications technology that comes after it, you can't really know what's happening simultaneously in many places all at once.

And it turns out the first step towards knowing what the weather is gonna be in one place at many times is knowing what the weather is at one time in many places. It's the kind of key to it. And so you end up as soon as the telegraph is invented, and as soon as there's a kind of rudimentary telegraph network, the telegraph operators begin sending messages to each other about the weather conditions. And they quickly realize that especially in the US, the weather is often moving from west

to east, and they can give some advanced notice of what's going to happen that afternoon, based on, you know, if you're in New York, what is doing in Ohio? And that just kind of basic sense that you could move faster than the clouds, that the news could move faster than the clouds, begins to open up this idea of a kind of holistic view of the planet. Suddenly, you can kind of imagine yourself looking down, not just on a map as a political idea, but really live seeing how the weather is changing over space.

And so in the 1840s, the Smithsonian Institute takes this sort of grand theoretical idea and turns it into an actual map, which is a kind of beautiful quirky analog fun thing that I loved your description of. Could you describe the map and how it functioned? Yeah, I mean, as with any corporate or government headquarters, when they built their new building, the centerpiece in the lobby of the Smithsonian Institution on the Mall in Washington was a big map of the, you know, fledgling United States up on the wall, pre-civil war 1840s. And whenever they got a report in from their

Smithsonian observers, their kind of brand new network of weather observers, they would put a little paper disc up, and the disc would be, have the temperature, it would be, have a different color for the weather, so white for fair weather, you know, black for rain, brown for clouds, blue for snow, and so when you arrived at the Smithsonian, you could look up at the wall and you could see what the weather was across the country. And you could begin to have that first inference of the weather of the future, you know, the forecast based on how those patterns might be changing. Yeah.

So then we get to the 1870s when there's an international coalition forming to expand the weather forecast. And people are starting to think about how to collect and share weather data more widely. I mean, from the beginnings of essentially international networks of any kind, you know, in the 1870s, you have international telegraph networks, the postal union is formed, you have the meter convention, you know, it's this kind of really vogue for standardization. And a big part of that is the recognition that if you have

brand new national weather services, they need a common language for communicating their observations with each other. And we're each going to maybe make our own forecasts, but certainly knowing what the sky is in your country is useful to my country. And that kind of basic sense of meteorology is a common good of the Earth's atmosphere is continuous. It really becomes part of meteorological culture

from the beginning. They are very good from a very early stage at cooperating with each other. Yeah, and so it becomes as much a diplomatic project as a scientific one. Yeah, yeah, absolutely. So how do people go from gathering data about the weather to actually doing something about it?

Where do they actually start to look into the future of what was to come? Well, the first person who kind of codified the process that has become the weather models as we know them today was a Norwegian meteorologist named Wilhelm Björknes. And it was in the 1890s that he first began to play around with the idea that you could treat the weather forecast as a hypothesis, as a kind of mathematical hypothesis, that if you could calculate the weather, if you could calculate the evolution of the atmosphere, you know, temperature, its pressure, its wind direction, and you could do that mathematically, then you could be quite sure the next day if you were right or wrong. And if you were wrong, you could begin to refine your equations and then do it again the

next day, or you could even go back and use the previous day's observations and calculate it again. Right. But the mathematical models, how complex are they and how far in the future can he really look at this point? Well his basic equations, which are now kind of known in meteorology as the primitive equations, how much I kind of love, his basic equations were right, but he couldn't solve them. He neither had enough observations, especially at different levels of altitude, and high up into the atmosphere, nor could he solve the differential equations required to sort of solve his own equations.

He couldn't actually plug the numbers in. So theoretically, he was mostly right, and in fact, the primitive equations are still at the root of the weather models. They're deep in there. They have evolved dramatically, but they're still there. They are still relevant. But practically, he got nowhere. He neither had enough to put into his math nor was he able to actually calculate what came out.

So then people can imagine these ways to get around this issue of the computation. So a mathematician named Louis Fry Richardson had this crazy idea that I want you to tell us about. Yeah. So, Bjorknis writes his paper in 1904, he says that we can predict the weather using math and physics. And about 10 years later, Louis Fry Richardson, an English mathematician, comes to it and says, well, I think I might actually give this a try.

And he actually uses a set of observations that Bureknaur himself would organize the collection of from a single day above Europe. And he begins this sort of furious six-week process of actually calculating that into a weather forecast. And he does it while he's working as an ambulance driver on the Western Front during World War I. Well, he was a quaker, so he wouldn't fight,'s working as an ambulance driver on the western front during World War I.

Well, he was a quaker, so he wouldn't fight, but he drove an ambulance, and so he talks about going back to his billet and sort of running the calculations with his slide rule and spending six weeks on this sort of single afternoon's forecast, which famously and spectacularly was wrong, sort of famous errors in meteorology. But he was convinced that if he had better observations tactically was wrong, sort of famous errors in a meteorology.

But he was convinced that if he had better observations and if he had a greater ability to actually make these calculations, you could have a useful forecast. And he comes to the idea that what it would really take would be 64,000 computers, which is to say 64,000 humans, human computers, arranged in a stadium. And there would be a conductor in the middle who would shine a light on them if they were going too fast or too slow,

and they would write their calculations and then pass it to the person next to them. And with 64,000 people, you could go fast enough to have a useful weather forecast, which is to say a forecast that is completed before the weather actually arrives. In one day. I mean, that's the thing. that is completed before the weather actually arrives. In one day. I mean, that's the thing. You know, you can have a very detailed forecast, but it's useless if the future comes before your calculations.

Yeah. He also somewhat amazingly predicts like the Google campus. He thinks that like his 64,000 computers should have like ball fields and cafeterias and entertainment and things like that. And he also predicts like kind of steampunk aesthetic as well. He describes these offices with like levers and desks and things that rise up into roof decks and basically what Facebook is in Manlopark today.

Yeah, that's so funny. So we have these people thinking in these big ways about the weather and how to forecast it. And we have these couple of limitations that are they're budding up against. One is computational limitation. The other one is kind of data limitation, like access to these measuring these points. So how was weather forecasting moving forward

in the rest of the world? And what were they doing to come up with what was going to happen in the future? When Richard Sint and Björknis, when their project essentially fails, there's this kind of amazing and pretty successful, basically 40-year history of meteorology, that actually makes a lot of progress, whether forecast gets better and useful and helps with early aviation,

you know, most famously, you know, the forecast for D-Day was a solid two-day forecast that allowed the allies to postpone their invasion. It's sort of always pointed at us as kind of forecast that changed the course of history. But none of it had anything to do with these calculations. It's sort of the equivalent of looking at a cloud or a cold front and just seeing it go across the country and it has not a little math in it, but it has a lot of just history and past precedent and stuff that lets you predict what's going to happen in the future. Yeah, absolutely. And it wasn't until the post-war era when you have the beginnings of spaceflight and the beginnings of real computing that the idea of actually functionally creating a weather forecast based on mathematical analysis of the atmosphere becomes possible again.

So after the war, we sort of get into the 50s and 60s and there's a big breakthrough. So new technologies emerge and there's a political will to build this whole earth map and make it really, really good. So tell us what happens in the 60s that makes jerkness's dream of calculating the weather finally come true. The most important thing is you have this kind of love affair with the earth, you know, with the earth as a planet. You know, you suddenly have this collective societal vision of what the earth will look

like from space. You know, you have all this science fiction. You have the first people orbiting the earth and everyone's sort of imagining what it is like to look back. And as soon as you kind of have that in the popular imagination, the idea of a map of the complete atmosphere becomes real. We go into space because whatever mankind must undertake, free man must fully share. There's this incredible moment in 1961, right after the Soviets first launched Sputnik,

where Kennedy gives a speech where he says, you know, have us put a man on the moon before the decade is out Provide the funds which are needed to meet the following national goals first I believe that this nation should commit itself to achieving the goal Before this decade is out of landing a man on the moon and returning him safely to the U.R. And that's point number one and turns out point number three is $75 million for weather satellites. Will help give us at the earliest possible time a satellite system for worldwide weather observation. Let it be clear. As familiar as that man in the moon line is, the line about weather satellites comes like 30 seconds later.

And for Kennedy, the global view of this was kind of part of the larger project of the triumph of American ideals around the globe otherwise. So you have this sort of moment where all of the kind of imperial ideals of a kind of American view of the globe and American dominance of the globe become wrapped up in a view of the atmosphere for scientific good, for meteorological good, for the sort of what we now think of as this banal project of creating better weather forecasts. So JK's vision came true in many ways.

Throughout the 60s and 70s, a lot of satellites went up into space, both for military surveillance and for weather forecasting. And as the weather machine grew, a worldwide alliance developed between nations. They figured out how to share data and how to maintain the infrastructure that they'd collectively build. The main part of the UN that now deals with weather is called the World Meteorological Organization, and they get together every four years to talk about policy, and Andrew went

to one of these gatherings. Yeah, in 2015 in Geneva, the World Minerological Congress is the big event every four years, and it's the world's weather diplomats coming together, and sort of methodically kind of hashing through their issues, and then breaking for receptions, which is the diplomatic word for party, as it turns out. and sort of methodically kind of hashing through their issues and then breaking for

receptions, which is the diplomatic word for party, as it turns out. It's mostly very specific and technical, but the dynamic between the countries that essentially run super computers and the countries that don't was increasingly apparent and not surprisingly, the effects of climate change are more pronounced for less wealthy countries, which is also the countries that don't fly whether satellites and run whether super computers. So there was really a sense that they were all in it together that this was a kind of thing that governments did.

And there was 150-year tradition of governments around the world sharing their data with each know, sharing their data with each other, sharing their forecasts with each other. And especially now, when storms are more powerful, when the effects of those storms are more pronounced, when there is this sort of growing threat of what will happen with the weather in the future, it was very clear that that cooperation

was needed now more than ever. There's this whole notion of the weather machine, this is this global project, it's carried out by governments, it's done for the public good, but increasingly, private companies are getting into the weather forecasting business. So tell us about that and how this is interacting

with the sort of global project that's been going on for decades and decades. Yeah, well, there has been the assumption, essentially, since the birth of satellites and computers, that super computers and satellites are things that governments do. They're too expensive for private companies to do. If you have a weather service, you know, and you need a $30 million computer, that's going to be something that a government buys and it's going to be in service not only to its citizens,

but to the entire world. But the couple of current circumstances are colliding. I mean, one, you have the couple of current circle are colliding. I mean, one, you have the sort of rise of private spaceflight. You have the kind of space access of the world, and you have private space observation companies. You have more severe weather and more money at stake to predict that weather.

And you have a kind of rise of the recognition of big data and what we can do with data and how important it is to understand the world using big data and what we can do with data and how important it is to understand the world using big data. And so you end up now with the idea that private weather forecasting is probably a pretty good business. And so after 150-year tradition of weather forecasting

being something that governments do for their citizens, there's now a bit of a gold rush where companies can run their own weather models, can fly their own weather satellites, can collect their own weather observations, and provide a private forecast that is a value that exceeds the usefulness of the publicly available forecast. And how do you think about that as someone who's seen the long view of the weather machine turning towards privatization? Like, what do you think are the complications like now

and maybe the complications in the future? Well, I mean, the first thing that I saw was the real angst among the sort of died in the wool government meteorologists over what this meant for the long tradition of government weather services protecting life and property. And that being something that governments do for their citizens. But of course, from a technical standpoint,

you have the possibility of even better weather forecasts. And so there's certainly a kind of technological thrill with the idea that this could be improved, but it's not hard to recognize the kind of global inequality suddenly appearing in the technology of the weather forecast itself, and this real exacerbation of the effects of climate change when you have hurricane forecasts accessible to the rich before they are accessible to the poor. When, of course, they will affect the most vulnerable more directly.

Yeah. We get to the sort of the crux of this, which is like, at this moment, being aware of the new extremes when it comes to our climate is more important than ever, we're at this moment where, you know, privatization and proprietary data and models could break that apart. And it wouldn't take much as the sort of strange thing. All the weather observations that are collected by the US government are put, it's kind of right

in the global bucket. And in exchange, we get all the world's weather observations that are collected by the US government are put, it's kind of right in the global bucket. And in exchange, we get all the world's weather observations back. And if, for example, the National Weather Service decides to buy private satellite observations for one category, and that company says, no, you can't share that, and that's big it has turned off, the possibility that other, you know, start with European countries will say, well, if you're not giving us that data, we're not giving you our data. And within two or three days, you know, the entire system falls apart. And it's not as if, well, we only need observations over the United States for forecasts over the

United States. As soon as you're passing a three or four days, you need that entire global view. And all of the weather models are kind of built on that holistic global view. And so the idea that this is a kind of, you know, this is within our borders, that this is a kind of local issue, that it isn't entirely international, interdependent, is preposterous. And of course, it's deep in the kind of global order

that the US built up in the second half of the 20th century. You know, it is the kind of American ideal of leading the world with technology and cooperation. And at the moment, and not only in the Trump era, but really over the last 10 years, particularly with the kind of new technological dominance of the US, with the Googles and Facebooks of the world,

that the idea of the sort of proprietaryness of this data as something that is deep in the heart of our system becomes more consequential in the way that we put together weather forecasts. Yeah. I think one of the things that's fascinating about all this and all the work that you've done and my thinking on it that has evolved since reading your book is this weird mixing of the idea of weather and knowing the weather being so kind of the idea of whether and knowing the weather, being so kind of the now and every day, and how much it's about little tiny decisions about whether you bring an umbrella,

and also about hurricanes, it kind of gets your mind reeling in this very strange way about just like the human desire to know what's coming. Yeah, yeah. This book took me several years to write, and in the course of writing it, my older child, my daughter, went from kind of a toddler to like a proper elementary school student. And at the beginning, I would be working on this

and she would say, what's it going to be tomorrow? Like that would be kind of her last words before going to sleep. She meant like, what are we doing? But I would be like, oh, how do you consider the future? What does it mean that the sky is coming this way? And I'm sort of rooted in place and time and how is this going on?

And so I kind of heard it as like, what's the weather going to be tomorrow? And that sort of contrast between my watch is going to tell me what the weather is going to be tomorrow. And that's just super easy and no worries. And then the existential dread of what's it going to be tomorrow was right there with it. It's the most banal thing. It is the ultimate small talk.

And yet it's also, of course, the core of our existential planetary dread. And this isn't some way the sort of parable of climate change as well. We can be pretty sure about what's going to happen and the ability to change it or to do something about it is completely independent of that foresight. In other words, we have the information.

We can effectively see into the future, but what we do with that information and how it is used for planning and preparation is up to us. To find out more about Andrew Blum's book, The Weather Machine. Go to nyanipi.org. We're going to visit a tiny island in the North Atlantic that is a tiny cog in the gigantic weather machine after this. One of the most fascinating things about the entire system used to protect the weather is how reliant it is on space aid satellites orbiting the Earth and hundreds of more humble weather stations located on the land. Both technologies are needed to inform our global view, because the weather machine is so vast and made of so many parts, no one thing exemplifies it all.

But I asked Andrew Blum if he could zoom in on one of his favorite places that's essential to the whole data gathering apparatus. When you try to kind of peel open the weather machine and see what it's made of, you end up with this kind of challenge of choosing a single place to represent all the places, which of course is kind of impossible. You know, it is the nature of places that they are all different, so they all occupy a kind of different spot in the map, partly because of Birkenis and partly because of this Norwegian meteorological tradition,

I latched on to Norway's system of weather observation. And in fact, I fell completely in love with this island called Yan Mayan. That's this Arctic island way off, kind of towards Greenland, that only has a weather station on it with an army crew, they get serviced a few times a year and a couple huskies, and it sounds like this really kind of incredible wild place. Which means this to say you can't go there,

if you go, you have to go for three months. So I kind of abandoned that dream of actually visiting this weather station, but found instead a place called Ducira, which is a sort of small island off the coast of Norway, and when I say off the coast of Norway, I mean, it's like a 25-minute ferry ride, like, you know, no big deal, it runs a few times a day, but because of its location in the North Sea, it has been an important weather observation point, basically for 150 years. And so you have a very early telegraph line there, and you have a single spot kind of up on

the top of the hill in the center of the island that consistently has been the point where the Norwegian Weather Service has observed the weather. It's a very windy place, which is known for its birding and for its winds. And when you're there, you realize, I realized what it meant for the kind of win to be rushing by this single point. And I realized that that's kind of what win is. You know, win is the passage of the atmosphere, past a single spot. And you know, that has to then be tied back into the kind of global computational system.

You know, you need to sort of send word back to Oslo and Oslo needs to send word back to Frankfurt where the sort of European collector is, and then that gets sent to Virginia, and the entire thing kind of gets networked together into typing in Utsiro and Google, and then the temperature shows up. But all of those things have to fit together, and that system has to be deliberately designed,

and the kind of design of that system goes back to the middle of the 19th century with the sort of first recognition that not only was it useful to know what the weather was in other places, but it was suddenly technologically possible to get that news pretty speedily. Right. And it's gathered by a human or tended to by a human, you know. It's tended to by a human. Yeah, he runs the restaurant and he does the weather observations.

So it's four times a day, he goes on his back stoop and he has a cigarette and he kind of looks at the sky. And then he goes to his computer and he logs in and he in the kind of Norwegian weather services drop down menus. He sort of does a qualitative analysis of what the clouds are according to the sort of rules that he's been taught. And that gets then sort of put in the whole way. And the same thing happens in every airport, in the country, every major airport has around the clock weather observer, some person who's in an office somewhere on the grounds of the airport, who once an hour is checking

the observations that the automated system has made to make sure the system's working, and if the clouds are slightly different than the Celieter, the cloud observing machine can read to correct those. Andrew Blum is the author of the Weather Machine, a journey inside the forecast. 99% of visible was produced this week by Delaney Hall, Mix and Tech Production by Sheree Fusef, Music by Sean Riel.

Our senior producer is Katie Mangle, Kurt Colstead is the digital director, thrust the team as Emmith Fitzgerald, Vivian Le, Joe Rosenberg, Chris Baroube, Avery Trouffleman, Sophia Klotzger, and me Roman Mars. We are a project of 91.7K AW in San Francisco and produced on Radio Row in beautiful, downtown, Oakland, California. 99% invisible is a member of Radio Topia from PRX, a fiercely independent collective

of the most innovative shows in all of podcasting. Find them all at radtopia.fm. You can find the show and join discussion about the show on Facebook. You can tweet at me at Roman Mars in the show at 99PI or on Instagram and Reddit too. But we should really talk about the weather at 99PI.org. Radio Topea.

From PRX. Radio Tepi-A from PRX.