Advent of Computing - Episode 29 - PCM, Origins of Digital Audio

Episode Date: May 3, 2020

Every day we are inundated with digital audio: phone calls, music, even this podcast. Digitized sound has become so ubiquitous that it often fades into the background. What makes this all possible is a technology called Pulse Code Modulation, or PCM. This isn't new technology, its roots trace all the way back to 1937. So how exactly did digital audio come into being well before the first digital computers? Like the show? Then why not head over and support me on Patreon. Perks include early access to future episodes, and stickers: https://www.patreon.com/adventofcomputing Important dates in this episode: 1937: PCM Developed by Alec Reeves 1941: Germany Cracks A-3 Code 1943: Bell Labs Develops SIGSALY(aka The Green Hornet) 1957: First PCM Synthesizer, MUSIC I, Programmed by Max Mathews

By the year 2000, pulse code modulation in some form will be the very backbone of the world's communication systems that are internal to national or still larger units. End quote. In our daily life, we come into contact with digital audio almost constantly, whether we know it or not. Right now, you're listening to my digitized voice. It's encoded, packaged, and then delivered almost instantaneously straight to your ears. The last telephone call you made was, at least at some point, transmitted as digital audio. Even something as mundane as a ringtone or a message broadcast over a subway speaker, well, it's been digitized and then unpacked into sound at some point. digitized and then unpacked into sound at some point. And pulse code modulation, or PCM,

is the core technology used to turn sound into binary data and then back into sound again. The quote I opened with comes from a 1965 paper written by Alec Reeves, the man most often credited for the invention of PCM. Over 50 years ago, he accurately predicted a huge part of our modern day. But this technology wasn't invented in the 1960s. No, it's considerably older than that. Reeves first described PCM all the way back in 1937. Now, there's a lot to unpack in that. PCM is digital, but it was formulated well before the digital computer ever came into being. But perhaps more remarkable is the fact that this digital encoding technology, well, it didn't appear out of nowhere. The fact is that digital data was being ferried around at least as far back as the mid-19th century. The formulation of PCM can trace its roots to the telegram, a downright antique device.

And if we look at the larger context of audio recording, well, things like the vinyl record, CDs, even cassette tapes, they start to really look like a flash in the pan compared to the surprising longevity and amazingly widespread use of PCM today. Welcome back to Adren of Computing. I'm your host, Sean Haas, and this is episode 29, PCM, Origins of Digital Audio. Today, we'll be looking at one of those computer-adjacent technologies. As it turns out, the history of digitized audio goes back a lot further than you'd expect, or at least a lot further than I assumed initially. What makes it so interesting is that prior to computers ever existing on paper or in real life, very similar technologies were being used to handle audio data.

Binary isn't just as old as computers. And as you may have guessed from the introduction, this is another episode where analog versus digital is going to be a central focus, so I want to front load a little bit of that discussion. Since we aren't strictly in the realm of computing, things are a little different. The analog or digital distinction when we're talking about computers comes down to how numbers are represented and worked with. Digital ends up being advantageous for that because it's a good fit for the underlying electronics that make modern computing tick. But in the case of sound, there are some slightly different considerations. One of those is, of

course, how the final product actually sounds. Analog sound recordings, here I'm thinking something like a vinyl record, well, they're often toted for having much better audio quality than their contemporary digital counterparts. But the big issue you face with analog audio, especially when you're looking at transmitting that sound, has to be noise. Analog signals can get degraded depending on how they're transmitted, and noise can easily seep in. Digital audio doesn't have that issue. Digital signals in general are pretty resilient to noise and error. You don't have to worry as much about the signal to noise ratio, and it doesn't degrade over time or distance. But the final signal may not be quite as high quality as a good analog

recording. It's a trade-off, but in general, digital audio is a little bit more flexible. This podcast is a good example of how handy digital audio really can be. I record everything from the comfort of my own desk. My voice goes into a microphone directly fed into my computer. From there, I edit everything down, make a final file, and then post it online. In a matter of seconds, anyone around the world can be listening to the latest episode. And every listener gets the exact same audio signal, no matter where they are or what they listen with, or even when they listen. That's something that you can't easily do with analog in the same way.

So in the modern day, digital audio is handy. But its original purpose was a lot more serious. As it turns out, once you get audio into a digital data stream, you can do a lot of really interesting and useful things with it. Editing audio is just one example. So today, we're going to look at how sound was first digitized. Along the way, we'll see how pre-computer technology was used to handle digital audio, how this technology became critical to the allied

effort in World War II, and finally, we're going to look at how audio eventually transitioned into the computer. To tell today's story, we need to go back to the 1930s, so I think the best place to start is with the state of communications in that era. Radio was already a well-established option by then, first being used in the 1880s and pretty quickly gaining a lot of ground. Around the same time, the telephone appeared. Both of these technologies are remarkably similar. For instance, a radio system works by turning sound into an electric signal and then transmitting that over the air as radio waves.

A listener's radio then turns that electric signal back into sound. The entire system is analog, so overall it's not that complicated. An analog sound wave can be turned into an analog radio signal and then back to a sound wave without a whole lot of processing. It goes without saying that radio and the telephone have both been extremely successful. There have been improvements, but over 100 years later, radio stations are still broadcasting and people still talk on telephones, and systems today use really similar designs to their 1880s counterparts.

There are some counterpoints to this. Certain countries have transitioned to entirely digital radio signals, and nowadays many phone conversations are handled, at least at some point, as a digital signal. But by and large, the core technology used by these devices has remained relatively unchanged. You can grab a radio set made in the early 1900s and still listen to the news today. It's really a testament to how useful communications technology is. But that being said, there are some big problems with these analog telecom systems, and these problems were well known since early on in their development. Primary among these has to be noise. No matter how well you engineer things or how good connections are, analog signals are always susceptible to noise. It comes down to the

underlying physics of the problem. Here, I think a phone line is the best example to use since we don't have to think about radio waves as much. When you talk into a phone's receiver, a microphone turns your voice from a sound wave into what amounts to an electric wave. That wave then travels down a wire until it reaches a phone on the other end. Once there, it's turned back into a sound wave via a speaker. In theory, this should perfectly transmit your signal from one place to another, but the real world has some complications in store. As it turns out, wires aren't actually perfect conductors. That's true of all metals in general. Each inch of a phone line has some inherent resistance. So, over a long run of wire,

your original signal will become weaker and weaker until it's unintelligible. Then, there's also interference. When other waves interact with your intended signal, well, you run into some problems. This can come from a lot of different sources. In some cases, the telephone lines themselves can pick up nearby radio signals, or a phone on the receiving end can just handle the incoming signal a little bit poorly. Your signal is, after all, just an analog wave. It's just a series of higher and lower voltages. And there are a lot of other analog waves that run around and those can get mixed up with your intended message really easily.

For longer runs of cable or to just deal with weak signals, you can build an amplifier. This will boost the signal so it can travel further and be just louder in general. But it also boosts any noise that's gotten to the signal, since remember, it's an analog wave so everything is just mixed together. The net result is that the receiving end of things gets an adulterated version of your transmission. Or in some cases, nothing comprehensible comes through at all. No matter how much you dress up these analog systems, you will always have these sorts of issues. Noise and interference, that's just part of an analog telecom system. But analog telecom wasn't the

only means of long-distance communication in the 1930s. The other way to get messages around quickly was via the electric telegraph. Now, this first appeared as early as the 18-teens. The telegraph predates radio and telephone considerably. There are myriad different implementations of telegraph systems, and it gets really complicated if you try to describe any of them together. But all of them work on a common level by sending a message coded as a series of on-off pulses down a wire to a recipient. Now, that should sound familiar. Speaking broadly, a telegram is set up to transmit binary data. I mean, it's not quite the same type of encoding

that we'd call binary today, but it works on the same principle. And since the telegram is in the realm of digital data, it has all the expected upsides. The transmission is less susceptible to noise. You either have an on or an off. Anything in between, you can just throw away. There's no ambiguity like in an analog signal. It can be amplified without the addition of any noise since you can exactly duplicate the signal. But to get all these benefits, you end up restricting what can actually be transferred. Any message has to be encoded when it's sent, and then decoded once it's received. Systems like Morse code do this by turning each letter of the alphabet into a series of long and short pulses of on and off.

It works, but you can only send what you can encode. So therein lies the problem facing telecom in the 1930s. There were means to transmit audio, but radio and telephone systems had issues with long-range communication and noise in general. On the flip side, the telegraph could send a message without error or interference posing a large risk. On the flip side, the telegraph could send a message without error or interference posing a large risk. But the message's contents had to be relatively simple. At the crossroads of these two technologies is where we find our way back to the engineer I mentioned at the top, Alec Reeves. By 1930, Reeves was an engineer working at International Telephone and Telegraph Research Lab in Paris. A few years prior, he had helped lay one of the first transatlantic telephone cables, a major feat in and of itself.

He was at the cutting edge of telecom, and therefore, he was intimately familiar with all the issues facing current technologies. With his work on transatlantic lines, he became very familiar with the issue of noise amplification. Since analog systems can only travel so far down a run of cable, the initial transmission would have to be amplified multiple times on its trip from Europe to North America and back. Each time, the signal was boosted and more noise would be added. So while a phone call from Amsterdam to New York was possible and it was intelligible, it wouldn't sound all that good. Telecom engineers of the day were pretty eager to find a way around this problem,

and Reeves was no exception. But here's where things take a bit of a turn. Reeves didn't try to make a better telephone, and he didn't really try to improve the telegraph, at least not on its own. Instead, he started looking for a way to put these two technologies together, taking just the best aspects of each device. And in 1937, he'd file a patent for his new creation, a device that was capable of, essentially, transmitting audio as a telegraph, or something near to a telegraph. The way Reeves managed this was inventing a technique called pulse code modulation, or PCM,

and it's actually a really elegant way of handling data. Essentially, PCM turns an analog wave into a stream of binary data. This is accomplished by sampling the amplitude of the wave, or how loud it is, at even time intervals. The analog loudness gets turned into a digital number and then pushed out over a telegraph line. What I think makes this so cool is that PCM is able to reduce audio down to a single variable – just how loud the sound is at a given time. This way, you can turn the sound wave into a set of numbers and then turn those numbers back into a sound

wave. Reeves describes this encoding system in full in his patent, but he goes a step further and lays out a full description of a device to encode and decode PCM audio. The core of this machine was an analog-to-digital converter, also known as an ADC, and I think this is a component that deserves a little bit of a special focus. As the name suggests, an analog-to-digital converter takes an analog value, in this case a voltage from a microphone, and turns it into a digital value. It's a really simple concept, but an ADC is one of those devices that shows up almost everywhere. In modern times, it serves as a bridge between computers and the outside world. But for Reeves' purpose, it was a way to transform a sound wave into a digital

approximation. In his circuit, the ADC was set up so that it would sample an incoming analog signal hundreds or thousands of times a second, convert that sample into pulses of binary data, and then send that down a cable. On the other end, the process would go in reverse. A digital-to-analog converter would sample the incoming binary data, turn it into an analog voltage, and then that fluctuating voltage would drive a speaker. turn it into an analog voltage, and then that fluctuating voltage would drive a speaker. The final piece of his patent addressed the issue of amplification. Now, despite being in a fancier digital form, a PCM signal was still just a stream of electrons. Over a long enough cable, it would dissipate, just like an analog signal. And like I mentioned earlier, amplifiers were already in use on transatlantic cables, but these were analog amplifiers.

To make the entire system work, or work as well as possible, Reeves described an equivalent for digital, something like a digital amplifier. A normal amplifier is, for lack of a better term, pretty dumb. It simply takes an input voltage and then boosts it to a larger output voltage. In contrast, Reeves' device, called a regenerative amplifier, is a much more complex affair. It works by reading the input signal, which is digital, and then reconstructing it at a higher voltage. By essentially rebuilding the binary signal, no new noise is added. Overall, Reeves describes a fully new system for transmitting sound. His implementation of PCM doesn't use any markedly new technology,

but instead, it repurposes existing infrastructure and technology to make something totally new. Despite how impressive this new creation was, it wouldn't take off overnight. As it turns out, the circuitry to encode and decode PCM is pretty complex, especially by 1930s standards. This was compounded by the limited technology of the time. Transistors didn't yet exist, so Reeves' design relied heavily on vacuum tubes. The main issue here is that vacuum tubes are relatively slow to switch. They're not all that cheap, and over time, they're pretty unreliable. Reeves hit a wall pretty quick, where it was determined that his machinery was just too expensive to warrant all that much more progress.

Timing also played a factor. As the 1930s turned into the 1940s, Reeves had other things occupying his time. To quote, having had it patented, for understandable reasons, I then let the invention slip from my mind until the end of the war. End quote. If it wasn't for World War II, perhaps Reeves could have spearheaded faster development of PCM. Maybe with a larger push, the method could have been adopted. But just as war would push Reeves away from PCM, it would also pull another group of engineers into the fray. A large factor in running an effective military is communication. And in wartime, that factor becomes even more important.

communication. And in wartime, that factor becomes even more important. You need a way to connect and coordinate your military quickly, but whatever method you use, it also needs to be secure. If the enemy can read your dispatches, then you're in for a world of trouble. Encryption is one way to solve this problem. By scrambling and encoding your communications, it's possible to keep it safe from prying eyes. And during World War II, encryption would become its own arms race. Famously, the German army used the Enigma machine to secure their communications, and it's a great example of effective use of encryption. Enigma took a plain text message and an encryption key, one that was updated daily and known only by the German

military. The message was then passed through the machine, and a scrambled string of letters came out. If you had an Enigma machine and you knew the proper password, then it was a simple matter of turning that jumbled message back into the original string. But, to anyone else, the encrypted text was utterly meaningless. It would take a massive amount of work on the part of the allied powers to find a way to crack Enigma's code, and by doing so, the Axis lost their ability to communicate covertly. While Enigma is the most well-known, this back and forth of making new encryption schemes, those codes being cracked, and then going back to the drawing board, it would play out throughout the war. One of the encryption systems that was tried, cracked, and subsequently fell out of use was

called A3. This system was used by the Allied powers as a way to conduct secure phone calls up until 1943. It had the same goals as Enigma, but it went about things in a totally different way. First of all, A3 was an audio system. Instead of encrypting a string of characters, it worked by scrambling an audio signal. It was also a purely analog system. It worked by taking an input analog signal that was then broken up into different frequency bands. Those bands were then rearranged, sent, and on the other end, unscrambled. To anyone listening in, the call would sound totally unintelligible. But if you knew how it was scrambled, then you could rebuild the original sound wave. This worked for a while, but it couldn't last. By 1941, German engineers

had already figured out how to listen in on A3 calls. This was kept under wraps, but it was only a matter of time before the Allies realized that their communication lines were no longer safe. A big reason for A3's failure came down to availability. The device had been developed by Bell Labs prior to World War II, and they had been selling units around the world. By the time the war was declared, the German government had their own and had been using A3 for a number of years. All they had to do was figure out the settings being used to scramble the signal and boom, they were in business. Once the security hole in the system was realized,

it was already far too late. In 1943, development on a replacement would begin. Once again, Bell Labs was contracted to devise a solution, but this time they were aided in their work by none other than Alan Turing. During the war, Turing was instrumental in cracking the Enigma's encryption, but he was also working on the other side of the puzzle. Turing, his colleagues at Bletchley Park, and Bell had already been looking at ways to encode voice communications. For Bell, this took the form of the vocoder, a device that was able to break down a voice into a series of characteristic frequencies. This was similar to the technology used in A3. Once encoded,

a vocoder signal could be transmitted and later reconstructed. Concurrent to this, Turing and developed a similar system called Delilah. Instead of focusing purely on encoding, Delilah was a machine made to encrypt voice transmissions. Neither project was advanced enough to replace A3 on their own, but it gave both teams the background needed to tackle the problem from a better direction. The key to developing a better voice encryption system was finding a good way to turn audio signals into something that was more conducive to encryption. Encryption works best with discrete data. That's one of the reasons for Enigma's success. Letters are discrete,

a voice, not so much. And this is where pulse code modulation comes back into the picture. And this is where pulse code modulation comes back into the picture. Shortly after work started, the team of allied scientists stumbled upon Reeves' PCM patent, and they realized they could solve a lot of their problems. The system they would produce beautifully illustrates another important feature of PCM. The final machine was called SIGSALLY, all caps, S-I-G-S-A-L-Y. Personally, I prefer one of its early codenames as I find it more fitting. That's the Green Hornet. It just sounds a lot more cool, and it's a little bit more descriptive. To anyone without the proper equipment, the encoded data stream was

said to sound like a buzzing hornet, and the Green Hornet comics were popular at the time, so why not call it that? Anyway, the machine would incorporate a combination of Bell's earlier vocoder, PCM encoding, and a layer of stronger encryption. To start with, raw audio would be passed through a vocoder to produce 10 separate signals. Each signal was a component of the original audio stream at a certain frequency range. Each of these new signals was then transformed into PCM. So now you have 10 streams of digital data.

Then that pile of numbers was run through an encryption algorithm. Since the data was, well, just data at this point, much stronger encryption methods could be used. On the receiving end, the process was just reversed. To know how to unscramble a message, you had to have the encryption key, the method of encryption, and how all the different signals were packed together. Really, it adds more and more layers to the security of their signal. This is a fabulous example of where PCM really shines. Once everything's mixed down to a PCM signal, it might as well have just been any other type of data. That means that you can do whatever you want to it without loss of fidelity. Unlike analog audio, every operation you record, with a few exceptions, is fully reversible.

A system like A3 could scramble and unscramble an audio signal, but the unscrambled signal will always have some added noise thanks to how analog audio functions. So there's a limited amount of scrambling and coding or just operations you can perform to an analog message. But with PCM, well, you can do as much encryption, ciphering, scrambling, unscrambling, and decrypting as you like. And it's all reversible on the other side and you get the exact same signal. For the Green Hornet, that enabled much, much stronger encryption. And by July of 1943, the first communications would start being sent over encrypted PCM.

So, why is it that the Green Hornet was actually built whereas Reeves' earlier inventions had been dead in the water? Well, it all comes down to resources. Secure communications were integral to the Allied war effort, so to that end, money was thrown at any promising project. And there was a large team working to make Green Hornet a reality. Those were two factors that Reeves didn't really have back in 1937. But just because there was a lot more research behind this new PCM system, don't think it perfected the technology. Not nearly. All the issues that Reeves faced with his initial system were still present. In fact, they were amplified. The transceivers used by Sig Salley were massive and complicated machines. Computers wouldn't exist until the very end of the war, and practical

transistors were still decades or so away. Everything had to be done in hardware, and pretty limited hardware at that. Each installment was built from vacuum tubes, essentially a scaled-up version of what Reeves had envisioned. Once you got together the vocoder, a PCM encoder, PCM decoder, hardware to handle encryption and decryption, well, you're looking at about 50 tons of special-purpose equipment. And that's not being hyperbolic. These units weighed 50 tons. This is where vacuum tubes start to become another big issue. Tubes tend to run hot. To work at all, they need to reach a certain temperature. And with literal tons of tubes running at once, each acted like a tiny heating element, so you end up creating a very expensive, very complicated oven. And they just plain take a lot of power to run. So when all is said and done,

each Sig Salley installation had to be housed in a purpose-built room with specialized cooling and power hookups. It's not that dissimilar from the not-far-off mainframes. Despite the limitations, Green Hornet proved invaluable to the Allies. It would be used past the end of the war, up until 1946. In that time, its encryption was never cracked. By the time the system was decommissioned, it had been used for thousands of secure conversations between some dozen sites, ranging from the South Pacific to Washington DC to London. The D-Day invasion of Germany was planned and coordinated in secret using SIGSALI. Its importance to the story of World War II can't be understated. But despite being such a big deal, the Green Hornet was ultimately just another step. It was the first practical implementation

of PCM audio, but it had major faults. Purpose-built, massively complex, and just massive hardware was an obvious dead end. However, just as Sig Sally was decommissioned, a new avenue for PCM would open up. Up to this point, we've been talking about pre-computer audio. But once the 1950s roll around, we start to slip into the era of more practical computing. When you get down to it, computers and PCM audio, they go really well together. Despite being developed in the 1930s, PCM was totally digital, and that's the perfect kind of data for a computer to crunch. That being said, it would take a bit of a push for this new pairing to really take off. In 1954, Max Matthews would join up with Bell Labs.

We'd probably call Matthews a computer scientist today, but at the time, he worked as an electrical engineer specializing in computers. He had a strong background working on early systems in his college days, and from early on in life, he developed a fascination with music. Even while pursuing his PhD, he found time to play violin with his friends. Even while pursuing his PhD, he found time to play violin with his friends. This mix of skills and interests would end up leading him to working in the audio research lab at Bell Labs. At the time, Bell was a massive company, with departments ranging from telecom to semiconductor development.

They had a lot of money at their disposal, and their laboratories were working on the very cutting edge of technology. For the 50s, that meant that Bell was one of a handful of places in the country with access to a computer. But being on the cutting edge didn't mean that everyone at Bell was working on totally new technology. A large part of the company was telecom. In the United States, they had an almost total monopoly on telephone systems. If you picked up a phone and it was connected, well, somewhere there was Bell hardware behind that. But, by and large, these systems were still analog. One major project at the time was finding a way to turn their wartime PCM experiments into a practical phone system. The biggest target for this was the transatlantic communication lines. PCM was ideal for these types of long-running cables, and with a little work, it would be possible to pack multiple audio signals into a single transmission down a single wire.

The key here was finding a way to pack or compress speech down into less data, while still being able to unpack it into something understandable on the other end. The more compressed the audio, the more separate signals you can send down a single wire. There was some promising movement in this, but things were slow. These types of systems were still driven by highly specialized and highly complex machines. Systems like this were inflexible, they were tough to use, and it took a lot of work to design, build, and troubleshoot. Matthews wasn't working directly with PCM early on, but he had a front row seat as his colleagues struggled to make something useful.

He put it like this, quote, People, not I actually, would work for two or three years on a speech compressor, encoder, and then they would try it out and you couldn't understand it. So they would tear it up and start over again. I thought the digital computer had now developed so if I could put speech into the computer, we could program speech code in a few months and then tear it up and start over again. End quote. This is the big advantage of computers. Sure, you can do everything with special-purpose hardware and circuits, but that takes a whole lot more time and effort.

With a computer, you can just write up some software to handle the same task. Mainframes of that era may have also been hard to work with, but they offered a lot more flexibility than just pure hardware. Pretty soon, Matthews would start tackling the same compression problems from this new angle. But things weren't all that smooth from there. I said that Bell had access to computers, but those were not on their premises. It was still too expensive and early on to have a computer at your own lab. Instead, they rented computer time from IBM's main office in New York City.

To complicate things even further, the system they had access to didn't have the proper hardware to handle audio itself. Even with fancy software, you still need an analog-to-digital converter to feed in audio, and then a digital-to-analog converter to pipe it back out. The solution the crew at Bell Labs came up with was a little janky, but workable. At the lab, they built what amounted to a digital tape recorder. Audio would go into the device, get passed through an ADC, and then written to a digital tape that a computer could read. Then, the crew would drive down to IBM. The latest version of their audio processing software would be loaded into the mainframe,

and the input tape fed in. The computer would do its thing and, after a few hours, spit out another tape with their processed PCM audio. Then it was all the way back to Bell Labs, where the digital tape was fed through a DAC, and then the team could finally hear the results of their work. Work would proceed in this fashion for a while. Matthews and his team bouncing between IBM and Bell Labs, slowly building their speech compression software. But more important than the project itself, Matthews was showing that computers could be used to handle audio data. And that PCM really belonged with computers.

Anything that dedicated hardware machines could do, well, you could replicate that in software, and it could be done a whole lot more easily. It was groundbreaking work that really presaged our modern use of PCM. So, we've already seen how PCM was used for telecom-like systems as far back as the 1940s. Matthews' compression system was just an evolution of that. The details were different, but when you get down to it, the purpose was very similar. You put audio in, it becomes a digital signal, and it gets sent to another place and eventually turned back into audio. Those types of systems are useful, but that's not the limit of PCM's usefulness. Starting in 1957, Macs would push digital audio in an entirely different direction, and it was possible thanks to PCM Audio. This new project was simply called Music One, and it would be the first practical

synthesizer program. There were already examples of computers generating sounds, but all earlier attempts generated on-the-fly tones. Basically, they just used a computer to drive a speaker at a specific frequency. It worked, but it's a relatively simple and limited approach. Max's MusicOne program was much more sophisticated. His system originally worked by defining a set of instruments and then using those to build up a musical score all inside the computer. Once everything was set up, your score was ran through a computer which mixed all the sound together and produced a finished PCM rendition of your song. This is another case where PCM can do things that traditional analog audio simply can't. Since all your audio is converted to binary data, the computer can treat it like any other

information. Another breakthrough that made MusicOne possible was the digital audio recorder. The idea of recording PCM for later reuse didn't come along for quite a while, mainly because all early applications were so heavily focused on transmitting audio. But the crew at Bell Labs already had recording technology in the early 1950s. The machine they used for transferring PCM around on tape was essentially an early digital recorder. Early on, it was only used for shuttling around test data. But there's a lot of potential in this almost incidental technology.

It would end up being used by Macs as a general-purpose way to get audio in and out of a computer. And in the case of Music 1, this technology would be used to grab samples of instruments. It worked out to where Matthews was able to record himself playing violin, load that into one of Bell's mainframes, and produce totally new compositions. With access to a computer, it was pretty easy to mix and mash up samples into new music. But that's only half of what makes Matthew's

work so pivotal. His next big leap was to cut out the entire input side of the equation. Working with an existing library of recorded audio samples is a powerful technique, but it's also limiting. Max's violin can only ever sound like a violin. At this point, computers were already able to process audio data just fine. So why not have them generate the input data as well? I mean, PCM audio is just numeric data after all. This was the basic idea, to produce a program that could synthesize sound using just a set of parameters. Matthews would work on this problem for a number of years, and finally codify it as what he called a unit generator. To quote from Matthews would work on this problem for a number of years and finally codify it as what he called a unit generator. To quote from Matthews again,

The crucial thing here is that I didn't try to define the timbre and the instruments. I just gave the musician a tool bag of what I call unit generators, and he could connect them together to make instruments that would make beautiful music timbres. I think this is where we see the full potential of what PCM can offer. Sure, you can use it for faring around audio, or you can use it as an intermediary step to process audio. What Matthews did was totally different than that. He realized that you don't need analog to ever enter into the equation, well, except for when you finally pull the audio

out and listen to it. PCM is a powerful tool for working with sound, but it can also be a powerful tool for making that same sound. And that idea is encapsulated perfectly by the unit generator. In essence, it's a totally virtual instrument. Instead of recording samples from a real instrument, a composer just configures the unit generator with a set of given parameters. Once set up, a unit generator tells the computer how to generate a certain type of sound. It's a general purpose tool for synthesizing totally new audio. With a keyboard or just some time, you can program up a totally new type of instrument, one that only exists in the computer. Or you could create a unit generator to process incoming audio, shift it around, and then pass out a transformed

sound. You can even chain unit generators together to make more complex sounds. It's a flexible framework for synthesizing audio from scratch. That's the basic idea behind what Matthews was developing. It was a totally new way of looking at sound, and a totally new avenue of use for PCM. But what did this all sound like? Luckily for us, well, there are archival recordings of this. Here's a 1961 rendition of Bicycle Built for Two, arranged by Matthews for the IBM 7094 mainframe. so The The music language would go through a lot of revisions. Given the time period, that clip would have probably been produced using Music 3. Notice that during portions of that clip, there is more than one instrument playing.

Now that's not done by having two speakers or by having two computers. Max's synthesizer accomplishes that by cleverly using PCM. The program being used here would have had multiple unit generators, each defining a different instrument. Then, as the computer synthesizes those instruments into audio, it simply combines the PCM waveforms generated by each instrument. Just this one detail really speaks to how well-suited PCM is for this type of work. But that's not to say that Max had everything figured out in the 1960s.

This was still very much the early era of digital audio. And, well, it's plain to hear. While interesting, the computerized rendition we listen to is pretty simple. That mostly comes down to the limited hardware he was working with. Computers in the 1960s were slow, lumbering beasts. That's a fact that comes up time and time again on this show. But that slow processing time had some direct implications when it came to audio processing and synthesis. These machines just weren't fast enough to do real-time sound synthesis. When you get down to it, working

with PCM audio just comes down to mathematical operations, which is what computers excel at. But it takes a whole lot of math to synthesize audio from scratch. On these old systems, it could take over a minute just to synthesize a single second of audio. So there was good reason to keep the synthesis simple. More complex sound would require many more unit generators, more math, and therefore it would take longer to synthesize. It may have been limited, but Max Matthews' music synthesizer really did show what the future held. The engineers and programmers of Bell weren't the only people who saw it this way. The new computer music had a way of capturing people's imagination.

The clip I played earlier may have sounded familiar. If not, then here's a little bit more of the full audio. It will be your answer too I'm mad crazy All for the love of you It won't be a stylish wedding I'd best afford a carriage But you look sweet upon the seat of a bicycle Built for two You look sweet upon the feet of a boy who feels heart. Sometime in the mid to late 1960s, Stanley Kubrick would find his way to Bell Labs.

At the time, he was working on the movie 2001, A Space Odyssey. And he came to Bell Labs to get an idea of what a telephone system might look like in the far-flung 21st century. While touring the lab, he caught wind of the singing computer that Max and his colleagues were working on. Matthews had synthesized the backing track, as we've already discussed. The vocal component was also fully computer-generated, developed by another engineer named John Kelly. Kubrick was enamored by what he heard, the idea of a computer that could produce music lined up with his futuristic world that was being put together for the film.

The demo, or at least the idea of it, would find its way into Space Odyssey as Hal's parting song. Here's what I think makes this all the more interesting. The vocal part of the Bicycle Bill for Two demo was produced using a vocoder, and of course, the music was all done using PCM synthesis. That combination is the same technology that was used in the Green Hornet system, a purely military device. Shifted around a little, and given a bit of an update, the same technology that helped the Allied forces win World War II was also used to produce the first digitized music. Alright, that brings us to the end of this episode. PCM has a wide-reaching and enduring

legacy. In fact, it might be wrong to call it a legacy, since it's still very much with us today. It's existed on paper since the 1930s, in part inspired by even older technology. Its first practical application was a purely military one, encrypting voice communications, but over time, the same exact technology would find its way into early computers, and into the realm of music. Everything I've covered today is really just the early era of pulse code modulation, the opening chapter, so to speak. Over the next few decades, we can see PCM driving more and more innovations. By the end of the 1960s, digital recording would start its climb to prevalence in the consumer market. Eventually, the CD standard would be codified with PCM as its backing technology.

And over the years, Matthews' music synthesizer software would make its way out of the lab and into the hands of more musicians. Once again, I think Reeves put it best, speaking on the state of PCM in 1965. Quote, it best, speaking on the state of PCM in 1965, quote, Pulse code modulation has been a child with a long infancy. Except for certain military uses not described here, in application, it is still only in the adolescent stage, end quote. It's a remarkably simple idea. And in a lot of ways, it's a really simple technology.

But the implications of PCM are huge. It changed how audio can be used. It has opened up a whole new avenue that was unavailable to analog audio. And hey, it's even made this podcast possible today. Thanks for listening to Advent of Computing. I'll be back in two weeks time with another piece of the story of the computer. And hey, if you like the show, there are now a few ways you can support it. If you know someone else who would also like it, then why not take a minute to share it with them? You can rate and review the show on Apple Podcasts. And if you want to be a super fan, then you can support the show through adjunct of computing merch or signing up as a patron on Patreon.

Patrons get early access to episodes, polls for the direction of the show, and other assorted perks. You can find links to everything on my website, adventofcomputing.com. If you have any comments or suggestions for a future episode, then shoot me a tweet. I'm at Advent of Comp on Twitter. And as always, have a great rest of your day.