Advent of Computing - Episode 119 - The Batch!

Episode Date: October 29, 2023

This episode we are looking at a ghost of bygone days: batch processing! Before fancy terminals peppered computer rooms, before there was a microcomputer on every desk, there was the batch. In this non-interactive form of computing a user could wait hours, days, or even weeks to get a chance at computer time. Machines were kept well away from programmers, guarded by digital clerics. Why did such an arrangement exist? And did it ultimately help the programmer?   Selected Sources:   https://multicians.org/thvv/compatible-time-sharing-system.pdf - Compatible Timesharing System, 15th Anniversary   https://sci-hub.se/10.1109/MAHC.1983.10026 - Rykman on GM-NAA I/O   https://ethw.org/First-Hand:Operating_System_Roots - Operating System Roots

To a generation whose concept of a computer is founded on the Z80 chip, it may be hard to visualize a 704, or to comprehend the place it held in the public imagination as the type specimen of what a computer was, a collection of mysterious, hulking gray cabinets, approachable only through the intercession of the operator. In the specially built computer room, the operator set switches, pushed buttons, and examined panels of flashing lights, while his assistants attended various whirring, clanking, and chattering devices, rushing to and fro with stacks of cryptically printed paper, decks of weirdly punched cards, and reels of recondite brown ribbon, all to the background hum of the machine. Add a little incense and a few candles, and you could be forgiven for thinking

these were the rites of some oracular shrine. Everything about the 704, from the inscrutable mainframe to the glowing tubes in the glass-walled core memory case, proclaimed that this was a very complicated system, operated only by specially trained personnel, among whom programmers and other ordinary mortals were not numbered. In short, a computer was something that you simply did not sit down and fool around with. J.M. Grates, in The Origin of of Space War discussing the IBM 704 mainframe, specifically the computer that he was not allowed to touch down the hall at MIT. Welcome back to Spook Month on Advent of Computing.

I'm your host, Sean Haas, and this is episode 119, The Batch. Although it may as well be called No Interactions. If you're new around here, then let me explain. Each October, I celebrate Halloween on the podcast by producing some spooky episodes. Think of these as computer ghost stories of a sort. There isn't much that's actually frightening in the history of computing, so these episodes are dedicated to the things that computer users fear, or the things that frustrate us. In the past, we've discussed viruses, email spam, uncontrollable robots, and horrific computer games. This episode, we're looking at a very particular era in

computing, the era of batch processing. It may take a little setup to understand how scary this truly is. There are reams of horror stories that you can find online about the era of batch processing. So let me start at the bottom of this. When computers first hit the scene, they were purely research machines. The only folk that ever touched the first generation of computers were deeply familiar with the machines. As such, there wasn't a huge need to protect the machines. Sure, these computers were very precious resources, but everyone understood that. Only very select few were even near these computers in the first place. Now, back in the day, the actual output of these computers was

measured in so-called computer time. It's a way of discussing a computer's operational time and how it's managed. In the earliest eras, computer time was scheduled out so that machines would be running as much as possible. Access to machines was somewhat controlled, but once your allotted time slot came up, you would likely be face-to-face with the honest-to-goodness computer. There wasn't a layer between you and the machine. Computers of this time weren't exactly interactive, but hey, if you're one of the lucky few, then you did actually get to use a computer. You could flip the switches yourself. As things expanded, as more computers entered into the world and more users were created, we reach a problem of scale. Until recently, there were always more users than

there were computers. The whole drive for a computer on every desk was revolutionary in part because back in the day, there may have been a computer for every lab if you were lucky, or maybe for every college campus. So how do you share a computer between many multiple users? The big fancy solution that we eventually arrive at was timesharing. You get piles of terminals that were hooked up to one big computer, and you have very fancy software that switched off between tasks. By switching very quickly, it looked like the computer was running more than one task at once, and thus it could serve more than one user at the same time. That's the approach that's most well-known, and honestly, it's the coolest.

It allows for interactive sessions, for face-to-face kind of computer time, at least in a simulated and very controlled manner. It's very, very nice. But there was another way. Batch processing. This method was in vogue between the early spread of machines and the eventual adoption of timesharing. And, at least in my view, as a modern programmer, this was probably the worst possible way to use a computer. Batch mode is non-interactive.

You basically drop off a program to be ran, and then you wait around to get outputs eventually. In this regime, a programmer never actually touches a computer. They work through an intermediary, an operator. Their access isn't just scheduled, it's fully controlled and fully mediated. This is, of course, not fun. But how bad was it?

How annoying was the process of batch processing for the programmer? Well, dear listener, you're in for a story of tedium, pain, and mainframes. Today we'll be examining first-hand accounts to work out what it was like for a programmer that worked by the batch. Along the way, we'll be examining the technology that made batch processing possible, and why it was even done in the first place. Were there good reasons for programmers to work at arm's length from a computer? Well, let's try to figure that out. Before we get into the episode proper, I need to throw in a plug just one last time. On November 4th, I'm going to be speaking at the Intelligent Speech Conference. This is a history podcasting conference that's ran online every year.

This year's theme is contingency, which is perfect for me since that allows me to discuss one of my favorite stories in the history of computing. There'll also be other great speakers from a lot of podcasts you probably love, like History on Fire or Our Fake History, some of my personal favorites. If you want to attend, tickets are currently on sale at intelligentspeechonline.com, to attend, tickets are currently on sale at intelligentspeechonline.com, and you can use my promo code AOC to get a discount on those tickets. With that said, let's kick into batch mode and see what we can dig up in some of the earliest computer horror stories. I want to start off this episode by trying to get into the headspace of a programmer in this era. As for a rough time frame, we're going to be looking at the late 1950s into the headspace of a programmer in this era. As for a rough time frame, we're going to be

looking at the late 1950s into the 60s. Probably something like the middle to late 60s. Computers of that age were so different as to be, frankly, unrecognizable. The rough shape of things had been worked out. We had stored program machines by that time, so we had actual programs that could run on actual computers, but don't be fooled by that. The sheer logistics involved were different enough to make a computer lab take on, well, a very different meaning. It was in this time period that some of the first commercial machines made their way onto the market. The IBM 704 that Grates worked with is a good example of these early commercial machines. On release, the 704 cost an eye-watering $2 million. It could only run one task at a time. It was an improvement over research machines and other earlier computers,

but it was still costly and very limited. It had to be used wisely, and it had to be protected. To continue setting the stage, we need to examine who was involved with batch processing. Here I'm going to be pulling specifically from Compatible Timesharing System, Here I'm going to be pulling specifically from Compatible Timesharing System, 15th Anniversary Comparative Overview, a paper published in IEEE about the first timesharing system. The setting for this part of the tale is MIT's Computation Center, sometimes referred to as the New England Computation Center. This was a computer lab on the MIT campus that housed, count it, one IBM 704. As near as I can tell, this would have actually been the very IBM 704 computer that Grates was kept away from.

The goal of the center was to provide computer access to academics in New England. At the time, there were very, very few machines floating around. So this was a way to share a very special resource. From the text, quote, The computation center had opened in 1957, and by 1958 and 1959, it was overloaded. People in science and engineering were using computers more, which Fortran had helped facilitate. End quote. Here's the interesting part. Fortran was a really new language at this point. To us, Fortran is the epitome of antiquity, this ancient evil that occasionally rears its head. But in the 1950s, it was huge. One of the primary reasons that Fortran was developed was to make programming more accessible. Now, we are only

talking about a very specific expansion in accessibility here. Fortran was designed to allow non-computer nerds to use a computer. But basically we're talking just other types of nerds. It allowed scientists like physicists to create computational models. It allowed engineers to simulate their designs. It even let some chemists work in code. This made up a class of relative neophytes, of newcomers. These were folk that actually wanted to use the computer as a tool. They were also the furthest from the machine. Getting a little closer to the computer, we reach the staff itself. These were the sages of the new digital religion. But not all members of this priesthood were equal.

Now, I'm working off of MIT's computation center here. There may have been differences at other labs. From the sources I've scrounged up, there were layers to MIT's setup. These layers were meant to ensure that programmers would have access to computer time, and also that computer time was allocated efficiently. For this, we actually get some insight from Thomas Kurtz, co-creator of the language BASIC. In the late 1950s, Kurtz was working as a professor at Dartmouth College, handily located in New England. Kurtz was selected to be a liaison between Dartmouth and MIT for the computation center. In practice, this meant that he would shuffle punch cards and printouts between the two campuses.

The general setup worked like this. A programmer at Dartmouth would punch their code onto a stack of cards. Then Kurtz would take over. To reuse a quote that I've already enjoyed in a previous episode, Kurtz explains shuffling cards around this way during an oral history interview. Quote, then I would carry them down in a steel box, catch the 620 out of White River, get into Boston around 930, and take a cab to the MIT campus or something like that, submit the cards box of cards would be handed off to another cleric. That person, the actual staff at the computer center, would feed the cards into the computer at the appointed time.

This was all scheduled out on a table somewhere, meaning that the code from Dartmouth would only have a limited amount of time to run. Once the programs had finished, the inner cleric would dutifully collect the outputs and bundle them up. Back to Kurtz, quote, at the end of the day, I would pick up a bunch of listings that were the results, which are usually error reports. Then I would cart those back to Dartmouth. Every two weeks, I did this. End quote. For the programmer at Dartmouth, turnaround was two weeks. This is an extreme case, but even for programmers at MIT, turnaround times could be a day or more. You might submit a program

in the morning only to receive the outputs in the afternoon or evening. If you had received a pile of errors in place of the expected outputs, well, that was just kind of your luck. In most cases, you couldn't immediately resubmit the failed job. You'd have to wait for your next scheduled chunk of time on the machine and just kind of hope that the next run would be less error prone. This, to put it bluntly, is one of the worst possible ways to work with a computer. Not only does it take ages to actually get anything done, but it's also an awful way to learn. way to learn. This is, I think, a bit of a fun angle to look at, since we get contributing factors that kind of lead to a special type of hell. In this period, Fortran was still being adopted. It was still in the process of gathering adherents. Most, close to all of those new

adherents, hadn't programmed before. That was kind of the whole point of Fortran. So we have the situation where a lot of new programmers are trying to learn the language. It's been shown time and time again that the best way to teach is with instant feedback. The shorter it takes to find out if an answer or a program is correct, the better we learn. That's just how humans operate. a program is correct, the better we learn. That's just how humans operate. I personally cannot imagine that it would be easy to learn to program with a day or more of lag time between attempts. That's about as far from instant feedback as you can get. This also lands us at another problem, although this is a bit more philosophical, so stick with me for this. In 1985, Peter Naur, known for a pile of contributions to programming language development,

wrote this paper called Programming as Theory Building. If you want a full rundown on the paper, the folks over at the Future of Code podcast have a really good episode on the topic. Highly recommend. Anyway, I want to give the short story here. Naur proposes that programming is primarily a matter of theory building. That is to say, the programmer starts with the problem they're going to solve. They build up an operating theory of how the problem works, then they work up a theory on how they can solve that problem best. Now, I personally like this point of view a lot. It separates the code you write from the actual

problem you're solving. The theory is all inside the programmer's head, and the code is just a way to act on that theory, to test it. It's also a very personal way of understanding software. It's also a very personal way of understanding software. A program on its own is a somewhat inert object. You might have a line that multiplies some variable by two. On its own, there's really no meaning there. But to the programmer that wrote the code, the one that has the theory inside their skull, they know that line is where they're doubling interest rates or calculating the number of wheels needed for a certain number of bicycles.

How does this tie into batch processing and long turnaround times, you may ask? Well, Naur introduces another concerning concept in this paper. You see, theory can be lost. The easiest scenario to imagine here is passing a project to a new programmer. Let's say someone takes over my bike code. This new programmer is going over my code, and they get to a line where I just say i equals j times 2. They know what that code does, it says it right there. It multiplies j by 2 and sets i to the result of that calculation. But they might not know why I'm doing that. They would have no idea

that I was calculating the number of tires needed for a certain number of bikes. The theory was lost in the shuffle. That makes the code harder to work with and understand, since it's just a reflection of the theory that I personally have inside my mind. However, we don't need to even involve another programmer here. Let's say I'm at Dartmouth in the 1950s, and my program is lucky enough to make it into Kurtz's magical metal box. It comes back in a few days later with all kinds of errors and weird outputs, but that's okay. I have two or so weeks to spruce it up and send it back. I procrastinate, and I end up getting to it the night before Kurtz is going to go back down and submit the next job. I'm trying to figure out the problems, and I run into the line I equals J times 2.

and I run into the line I equals J times 2. It's been two weeks since I wrote that code, and I have to rack my brain to try and remember what I and J are supposed to be. I have lost the theory. In modern languages, we have a lot of tools for dealing with this. We have things like inline comments, where we can put explanations of code in the code itself that gets ignored by the computer. And we have nice variable names. I could have written tires equals bicycles times two. That would have helped me preserve some of

the theory. But Fortran, the old school of FORTRAN, messes this up. In the first version of FORTRAN, all fixed-point variables must start with the letters IJK, L, M, N, or M. Variable names can only be six letters long at most. And each line of code can only be 80 characters long at most. And each line of code can only be 80 characters long at most. Once you deal with formatting characters, it's a little less, but 80's the rule of thumb. So I could maybe write n tires equals n bikes times 2. That's passable, but will I remember the particular naming conventions of Fortran two weeks after I wrote this code? Will I follow those conventions correctly in the next version of my code?

I'd wager that a new programmer might not. The new programmer is thrust into this inhospitable world, and they only get feedback in fits and starts. If you want a horror story, then that's it. Fortran, especially early versions of the language, are already hostile enough. Imagine being an engineer and trying to learn this new computer thing with all of these factors, this limited access, these weird naming conventions. This would definitely lead to some sleepless nights. There were, however, some programmers that could breach that inner circle.

There were some that could actually touch the computer itself. This came in a few forms. In some cases, programmers that designed system software would work directly with the machine. They kind of needed direct access in order to maintain crucial software anyway. There were also ongoing efforts to push past batch processing into new forms of computing. One example of this is at the computation center itself. In the late 1950s, John McCarthy started developing a language that would be known as LISP, or the List Processing Language. A large part of that language's design was interactivity, which meant that he needed physical computer time.

He was able to get scheduled time on MIT's 704. But there were caveats. This was still scheduled, regimented computer time. Once his shift was over, it was back to batch processing. This was also something of a special case. For most programmers, computers were still protected by this new class of digital clerics. This leads to, I think, an interesting avenue. You see, there was actual software developed to assist in batch processing. Operators would have initially had to do everything manually,

watching the computer and waiting for one job to complete before loading another. But that wouldn't last for long. As early as 1954, programmers were trying to automate away part of the operator's job. Or, in some cases, part of an unfortunate programmer's job. This tale actually connects us up to the earliest operating systems ever devised, which, as a twist, I wasn't exactly expecting this episode. The connection here starts before the period of full-on batch tyranny. Before the classes of sage and demi-sages walked the mainframe room. It starts with the release of the IBM 701, one of the very first mass-produced computers ever created. Now, when I say mass produced here, I don't mean that IBM was churning out all that many machines, actually.

We're talking a grand total of 18. Not 1,800 or 18,000, but 18. There were 18 701 machines mass-produced. When you actually count up every machine, there were only 17 out in the wild running with clients. So, there aren't really that many of these machines, period. The start of the story comes from George Reichman writing in IEEE Annals of the History of Computing. Reichman worked for General Motors Research, aka GMR, in the 1950s. This was one of the history of computing. Reichman worked for General Motors Research, aka GMR, in the 1950s. This was one of the few companies that actually had a working, honest-to-goodness,

701. As he describes, this low-production run actually led to a weird situation. To quote, during this here's the deal. GMR was leasing their 701 from IBM. You didn't get to buy these outright. But then IBM was turning around and renting the 701 back. There were just so few computers that even Big Blue needed to look outside for some cycles. In theory, this sounds great, right? Leasing a 701 cost $12,000 a month, or around $16 an hour. That cost could be offset by selling computer time to other organizations. GMR would actually only need to rent out their machine for 40 hours a month to break even at those rates.

That is really a steal. But this rental affair was totally manual. All operations on the 701 were manual, after all. And while the charge may have been hourly, slots weren't necessarily an hour long. According to Reichman, computer time was broken up into 15-minute chunks. When your 15 minutes of digital fame rolled around, you would physically walk into the computer room. You were your own operator, perhaps with a little bit of help to ease into things. From Reichman, again, quote,

He mounted his own tapes and set up the card reader, card punch, and printer to carry out his computation. By the time he had his calculation going, he may have had only five minutes or less of actual computation completed, wasting two-thirds of his time slot, end quote. But wait, dear listener, it gets worse. Context really makes those five minutes of computer time a pretty bad time. This is before Fortran, so programmers would have been using either machine code or a very, very early version of assembly language or some other automatic programming system. They basically had to speak in tongues, is what I'm getting to. This was very error-prone.

So those five minutes could easily be ruined by bad code. Imagine getting your manager to authorize some computer time. You spend all those funds, and you wait for your time slot. Your program's already written and everything, and then it breaks. That must have really ruined some weekends back in the day. The core of this problem was in how to efficiently make use of limited computer time. There were tricks to simplify setup and teardown, that was a start. There were shared routines that could be carried out, switches that could be thrown in the same way for every job, but when you get down to it, everything was still manual.

Even with standardized and streamlined jobs, someone still had to sit with the 701 and make sure it was eating cards, blinking lights, and spitting out new cards. Someone had to be there babysitting the computer and making sure it was starting up new tasks. Someone had to be there babysitting the computer and making sure it was starting up new tasks. However you slice it, these computers just weren't able to run at full capacity. Resources were limited, and they were being squandered.

This was the state of things on the 701. It was just a limited machine with limited capabilities. And really limited numbers. But these were known issues. The best kind of issues. In 1955, IBM started leasing the 704, an upgrade over the older 01. This is where the rubber would really meet the road. To continue the story, I need to explain some things about how

old-school mainframes worked. It's also all the fun wrinkles that I run into when dealing with these old machines. You see, mainframes are somewhat modular. The 704 is a great example of this. The entire machine is actually made up of a number of different components that all come packed in separate gray cabinets. The core is, of course, the computer. That's the 704 module itself, or, as IBM called it, the Electronic Analytical Control Unit. That was the brains of the operation, but it wasn't everything. IBM also offered the 711 punch card reader, the 716 alphabetic printer, the 721 punch card recorder, 727 and 753 magnetic tape things, the 733 magnetic drum reader and recorder, the 737, and the 740 Outputter, the mind reels at all of these options. Each of these units was a somewhat separate machine. The 704 had circuitry and interfacing to control all of these units,

but some could actually operate independently. So why would Big Blue do something like this? Well, one reason comes down to upgrade paths. With this modularity, a client could lease into a 704 and just punch card equipment to start with. Down the road, they might figure out they wanted a printer or even tape drives. IBM was glad to help with that upgrade for a low, low monthly price. This is also a legacy of how earlier punch card hardware was leased. You didn't get one giant card machine that did everything. Instead, IBM would provide a suite of machines that each did one type of operation. You could get card punches that would let you type data onto your cards. Sorters for sorting, tabulators for summing and reducing data,

cards. Sorters for sorting, tabulators for summing and reducing data, multiplying punches for multiplying, and so on. In that way, an office could be built up from semi-independent modules. Cards would be moved from one machine to another by hand as data was churned and burned. This older approach also has some downsides. In the biz, we might call this technical debt. Here, I'm of course talking about the cardstock elephant in the room, the punch card. The 704 didn't just have an optional card module. It expected data to flow in and out as cards, at least in large part. That was just the practice of the time. At this point, the medium was a solid 60 years old, and it was really starting to show its age. Punch cards were not very data-dense. You could, if you were being savvy and doing some weird stuff, maybe fit up to 130 bytes on a single

card. In practice, it was more like 80, what with bite sizes how they were back in the day. Already, that is not very good. Now, it may sound stupid to even mention this, but these are physical cards we're dealing with. Punch card equipment is electromechanical by nature. While there's nothing inherently bad about being mechanical, it does come with its own set of considerations. One of those is speed. Each card takes time to process. That time is dictated by the physical reader mechanism. Take the card reader as a prime example. A hopper is filled with punch cards awaiting processing. A card is drawn from the bottom into the machine. The card is then passed through a grid of contacts which reads the pattern of

holes. The card then needs to be ejected into an output hopper. This card is physically moving through the entire production line. That takes time. The 711 punch card reader used with the 704 could run at a maximum of 150 cards per minute. Doing some quick math and using theoretical maximums of a nice 8-bit byte, we get 40 bytes per second. That's bytes! I mean, I know punch card stuff is slow, but not that slow. That means that it would take an appreciable amount of time to load in a program on punch cards. That also eats into your 15-minute chunk of computer time.

This is where we get into really the dark art of batch processing. This lag time is unacceptable because it leaves a machine idle. That costs actual money, both for whoever is leasing the 701 or the 704 whatever computer, and also whoever is paying for computer time. The whole name of the game here is to optimize that time and thus optimize cost, and punch cards aren't very optimal. Luckily, there were tools close at hand to fix this problem, and one would lead directly to the codification of batch processing. The trick was to exploit

another type of storage, in this case, reel-to-reel tape. The older IBM 701 could be outfitted with a tape drive for storing information. Now, this is another strange medium that we really only see on old mainframes. These drives munched on half-inch open-reel tapes, and they were intended to replace punch cards, at least in some capacity. Oftentimes, data was kept on tape, while code stayed on cards. An accounting department might use reels to track payroll, for instance, but keep their payroll program on a stack of cards. This does make good sense. Magnetic tape is a read-write medium. You can actually change what's on the tape. Punch cards, you can only write two once, so they work best for more static information.

But here's the thing. You could actually just load code off a tape drive. There was nothing stopping you. In fact, there was even a programmer doing just this back in the day. In the early 50s, there was this dude named Robert Patrick, and he was working at Convair. That company had the same issue that GMR was running into. Most of their computer time was wasted by repetitive setup. Patrick figured that one of the larger bottlenecks was actually the punch card reader. So he worked out a slick trick. He transferred his programs onto tape. This trick was facilitated by one of IBM's earliest languages called speed coding. This was something like a proto high-level language. It's sometimes just called an automatic programming

system. It gave programmers a simple instruction set that would compile down to machine code. a simple instruction set that would compile down to machine code. Speedcoding had an instruction for reading and writing to tape. So all Patrick had to do was write up some code for moving around his software. The details of this saga are covered in the book The Computerized Society. The short version is that Patrick's tape trick saved him a wild amount of time. Now, this would have only worked for some types of software. From what I can tell, Patrick was working with some kind of repetitive code. His programs, once written, must have changed very little. Thus, computer time would have been spent running those programs with different parameters or settings or inputs. I can totally see this being the case with simulations done at an aircraft company like Convair.

Crucially, Computerized Society explains that Patrick was able to get more slots for computer time than his co-workers. This was thanks to the tape trick. After initial setup, he didn't have to futz around with loading punch cards ever again. Tape ran much faster than cards. The 701 tape drive ran at 875 bytes per second. That's with the conversion to standard byte sizes. It's still not stellar, but that's much better than punch cards. Patrick had found a way to more efficiently use a 701. In 1954, around the time that the 704 was just coming out, Patrick moved from Convair over to GM. IBM's big new machine, with all its bells and whistles, allowed for a variation on Patrick's trick. You see, you could wire up these modules to transfer punch cards

directly onto tape. Since everything was modular, you actually didn't have to go through the computer itself. You just connected up peripherals and let cards fly. While this is happening, the 704 could be doing whatever it wants. It could even be running another program. This could also be done in reverse. Flip some wires, probably throw a switch or two, and data on tape could be punched onto cardstock independently. All this while the 704 can hum away. This is the root of what becomes known as the three-phase operating system. Patrick himself describes the process in a 2006 article titled Operating System Roots, to quote,

The peripheral operator would take all jobs that had been submitted and load them on magnetic tape. The tape would be physically handed to the mainframe operator in the next room. When the 704 was available, the tape would be hung and the operating system would take over. Patrick continues, As each program progressed, it would write output on a magnetic tape in binary for later translation to decimal and printing offline on the peripheral printer.

End quote. The three phases here, input, processing, and output, were carried out somewhat independently. As he explained, tapes had to be manually created and then physically taken over to the computer. But once those tapes were loaded up, the 704 could really just sing. It could just keep going. Operators would fill a tape with all submitted jobs, all batches, creating an automated schedule for the 704 to work through on its own. That way, the computer was never sitting around waiting for the next job to be set up. Once one job finished,

the next was ready to roll. It would go that way until the whole tape was exhausted. The only software needed for this whole operation was in the middle step. This program had a number of names. Patrick calls it CISRES in his 06 article, but it's more commonly referred to as GMNAAIO. That name comes in later once the program gets shared around and collaborated on, but since we're talking about the low-level and early stuff right now, we're going to stick with Patrick's convention here. SysRes was a very simple program. Its job was to make sure that, well, jobs flowed. The actual input tape wasn't just a long string of programs one after another. Rather, each program had a header, a little chunk of data at the beginning of the program that described the upcoming job. When the tape was loaded, SysRes would read in the first header, load the job into memory,

and then just let that program run. Once that job was finished, SysRes reset the machine, read the next header, loaded in the next job, and then stepped back again while that job executed. in the next job and then stepped back again while that job executed. This provided huge benefits over the earlier method of operation. Now the machine could be left spinning at, essentially, all hours of the day. Inputs could be prepared ahead of time, and outputs could be collected whenever was convenient. Human interaction with the computer could also be minimized. human interaction with the computer could also be minimized. This is where the batch was born and class divisions started to form. To quote from yet another of Patrick's papers, this one from 87,

We threw the programmers out of the machine room, hired professional computer operators, and standardized the communication between the programmer and the operator. The programmers caught on to the new method of operation rather quickly since we instruct the operators to proceed to the next job when there was confusion or something didn't work out. End quote. Programmers were now shielded from the computer itself, protected by this new class of professional operators. That's one way to look at the situation.

Viewed from another light, it looks like programmers were now kept away from their precious machines, made to pass their prayers through a system of digital clerics. The final part of this quote is especially spooky to me. When there were issues with a job, when something went wrong, the sages were told to just move on to the next one. Now, I get why this was done. The whole point of this switch to batch processing was to speed things along, to better optimize computer time. It would be a waste of time to go grab the programmer and alert them,

or give them a chance to fix the problem on the fly. That would be kind of silly in this new streamlined world. I get it, but at the same time, this is the origin point of one of the biggest gripes we've seen this episode. Many programmers would spend hours or days waiting for outputs only to receive error messages, and there was nothing they could do until their next time slot rolled along. This brings us up to an essential question, one that I don't think has a good answer. Was batch processing good for programmers? Now, I want you to actually sit and think about this one. My gut reaction to this has always been revulsion.

What can I say? I'm a hacker at heart. I want to get my hands dirty, flip switches, and be illuminated by the glow of vacuum tubes. I find that when I have such a quick reaction, it is at least worthwhile to take a step back and examine things. So here's the deal. Batch processing pulls the programmer further away from the computer. GM-NAAIO, SysRes or whatever you call it, facilitates this. It codified the practice in software for the first time. Now, according to Patrick himself, this idea was heavily inspired by the Gantt chart, itself a tool used to streamline and schedule tasks. That right there is perhaps one of the most boring facts I've heard in my life, but fair enough, that's straight from the source.

That has to be the inspiration. But I want to introduce some of my patented speculation here. There's something that's hiding in the shadows of any discussion about batch processing, something that I've never seen explicitly addressed. That something is abstraction. If you aren't a veteran programmer, then I'll forgive you for not knowing this term. It's dangerous and painful at first. The general idea with abstraction is to express something in more generalized or abstract ways. That definition itself is pretty vague, which is kind of the point of abstraction. So let me give an example. I think that will help.

Fortran provides a form of abstraction on top of a computer. A digital machine will only accept digital code. A machine wants instructions passed down in ones and zeros. The computer expects those instructions to be very, very explicit. To carry out complex operations, you have to go step by painstaking step, telling the machine where to store numbers, how to manipulate each bit in memory, or which conditional flags to set on the processor state registers. That's not very abstract. It's all very specific. Fortran was so revolutionary, in part, because it allowed programmers to speak to a computer

in more vague terms, in a more abstract way. speak to a computer in more vague terms, in a more abstract way. In Fortran, you can write out normal-ish mathematical equations, then tell the computer to just do it. Which steps should the computer take to clear the accumulator? Don't care, figure it out, not my problem. Which address should be shifted to allocate a region for the next integer value. I don't care, figure it out, Fortran. All those considerations are dealt with automatically, perhaps magically, leaving the programmer to worry more about the program and less about the computer. That's the whole point of abstraction. It protects the programmer from getting their hands dirty, from flipping switches, and maybe even from basking in the glow of vacuum tubes. Now, at the time, there was actually a lot of resistance to Fortran and abstraction in general.

Existing programmers, read, those who already had pretty grimy hands, didn't want to give up the fine-grained control that they had over computers. Programming and machine code can result in faster programs, after all. It took considerable effort to get existing programmers to adopt this new language thing. There was resistance to abstraction, despite many good arguments in favor of the practice. In the modern day, abstraction is considered a very good thing when it comes to programming. That's just the common party line amongst everyone who writes software. So here's what's been bugging me. Is batch processing a form of abstraction?

If so, does that mean that batch processing can actually help the programmer? Think about this for a second before just jumping to a conclusion. I've been sitting with this for a bit personally. Batch processing does pull the programmer away from the physical computer. In that sense, it protects a programmer from the day-to-day operations of the machine. It protects you from a lot ofto-day operations of the machine. It protects you from a lot of the minutiae of the logistics. The whole point of SysRes was to remove the need to manually set up a job. That was a repetitive task that no one liked and no one really needed to do.

That said, I'd wager I could find accounts of programmers opining for the days of manual operations. Batch processing could also make a computer more accessible, at least kind of in a roundabout way. Think back to Kurtz's big box of code that he shuttled between Dartmouth and MIT every two weeks. Batch processing was the reason MIT was able to offer computer time to nearby universities. The very automated nature of this type of computing meant that a single machine could be used more efficiently. Time could be allocated better, more jobs could run, which meant that more programmers could get computer access, at least a limited form of access. access. At least, a limited form of access. Abstraction, in some cases, can fill a similar role. By streamlining a process, good abstraction can let you get more done. It saves time and it

makes work more efficient. So, in a way, we can look at batch processing as a weird type of abstraction. I think I'm comfortable saying that much. Abstraction and benefits aside, this takes us to another fork in the road. How does this practice go from General Motors research all the way up to Graetz's own nightmares of digital clerics? How does a fancy automation program lead to a generation of programmers crying outside the door to a machine room? Well, it starts with an innocuous word. Share. I feel like this episode is kind of starting to loop back into last episode's topic, and I swear, that wasn't intentional. I'm not smart enough to plan out things that far ahead. It might just be the magnetic pull of the season.

So let me explain a few things here. Prime among those is the user group. These days, a user group may sound like a quaint concept. The idea is to get a bunch of folk that all use the same hardware or software, put them in a room, and make them talk to each other. software, put them in a room, and make them talk to each other. Traditionally, it's over food or drinks. The first of these groups started rather informally. Back in the early 50s, during the 701s run, programmers started to talk as they're wont to do. In some cases, IBM actually facilitated this with conferences and informal get-togethers over dinner. The big topic of conversation always would go back to software. Or, rather, the lack of quality software. Each company that ran a 701 had found themselves developing very similar tools. Everyone needed improved code for dealing

with tapes and cards. Everyone wanted a better assembler or linker or some such programming tool. And, in a strange twist, everyone was actually modifying IBM's supplied programs to make them more useful. Put enough of these programmers in a room and they don't just talk, they start to scheme. Over the next few years, these schemes would solidify. One of the first was actually spurred on by the upcoming release of the 704. An outstanding issue of the older 01 was programming the thing. There were some diehards that liked talking to the machine in its native tongue, but the issues with this practice were starting to show. It just takes too much work and leads to a lot of errors. This was all

happening before Fortran, of course, so there wasn't a readily available high-level language to use. In 1954, a group of IBM users decided they would do something about this shortcoming. This group, including some programmers at North American Aviation, PACT. While not entirely a new language, it's also not not a language. It was called an automatic coding system, and it falls into this class of proto-compilers that appear during this period. PACT was first developed for the 701.

It got some use and was then ported over to the 704 as that new machine came out. This served as a great proof of concept. A loose group of IBM users were able to band together to work on a large-scale project. The resulting code was then shared between the group, collectively saving a whole lot of work. It was a beneficial arrangement for all parties. In 1955, this group was codified as an actual organization called SHARE. Technically, the name is in all caps, but it's not short for anything that I know of.

Also to note, SHARE is technically an independent organization legally separate from IBM, but there has been some crossover and help between the two legal entities from time to time. During the next few years, Share would grow. By 1956, there were 46 member organizations made up of companies and labs that were all running IBM hardware. Share organized conferences and collaboration, and they also provided a way to share new software. organized conferences and collaboration, and they also provided a way to share new software. Hence, the name. All members were invited to add software to Share's archives, which were maintained by IBM, and that archive was ever-expanding. Perhaps most importantly, Share provided a set of standards for configuring and operating IBM hardware.

This was especially important during this early period. IBM was still kind of transitioning from punch card hardware to actual honest-to-goodness computer systems. As such, much of their equipment was still configured using plug boards. These were messes of holes and patch cables that were plugged up to control old punch card equipment, and they were still present in many of IBM's early computers. This included the 704 and its peripherals. Scher passed around documentation on how these boards should be configured. This set a standard that allowed software from one member to run on another's computer. It's in this era of collaboration and standardization that Patrick comes into the picture.

this air of collaboration and standardization that Patrick comes into the picture. As a member of SHARE, he decided it would be prudent to show off his new three-phase operating system. I presented that plan at SHARE number three. As I gave that presentation, a SHARE fellow came up and talked to me. His name was Owen Mock. Owen said, we at North American have been thinking right along the same lines. Let's do it together. So his boss and my boss worked out the details, and Owen and I did what we now call architecture. And there we were programming on the GM side and programming on the North American side, and after we each got our half of the system programmed and checked out, we exchanged our portions and we had a system. End quote.

This would have happened around 1956, shortly after Cher was established as an official body. The North American here is North American Aviation. The Cher org codes for those two involved companies were GM and NAA, respectively. Hence this new system's name, GM-NAAIO. The operating system would reach its final form that very year, in 1956. And really, compared to other systems, it's a very simple program. From there, GM-NAA was passed around the share network.

I've seen it suggested that as many as 40 computers were running the operating system. But there's a distinct lack of citation for that number. Just know that this new operating system was very successful. It was well used and apparently well loved. And thus, we reach the next iteration of this cycle. This part of the saga is outlined in the wonderful tome, IBM's Early Computers. And I do mean tome here, this is a 700 plus page book, and it's quickly become my reference anytime I talk about IBM. By 1956, there was already a new computer on the horizon, the 709. The machine was a number of years out, but it was coming. Cher, as was becoming custom, needed to prepare. A committee was put together which included IBM

users and IBM employees decked out in their fancy blue suits. This committee worked out a list of software that Cher wanted to get ready for the new computer. This included the usual suspects, programming tools, assemblers, language support, all the stuff that programmers love, you know? All things the chair was already supporting on the 704. The list also had a new program on it. Or at least, it pulled a program deeper into the user group. The committee decided that Share would create an operating system for the 709. That operating system would be based off GM NAA IO. This new system would become a core project under Share. It was so important to the group, in fact, that it was named the Share Operating System, or SOS. And here's where things get a little weird. This is all happening before a single 709 computer had been delivered.

Share had insider information about the 709 because there are IBM employees that sit on committees. IBM works directly with Share, since it's a group of most of IBM's computer clients. Share itself worked up what they wanted the new OS to support. They worked up a specification. According to IBM's early computers, BigBlue itself offered to write the system to that spec. spec. But that led to a bit of a problem. You see, IBM had never written an operating system before. In fact, only a few people in the world ever had. Thus, IBM didn't have the experience they needed. The only folk who had ever done something like this were share members. So, for this weird chunk of time, IBM employees and IBM clients ended up collaborating on a project under the auspices of share. The result of this was an almost first-party operating system for the 709.

SOS was initially released in 1959, and it quickly became the system of choice for share members. This was, of course, an operating system designed for batching. So share became a batch shop. It's from here that IBM becomes a batch shop as well. At least, that remains a large focus for the coming years. During the same time period, FMS, the Fortran Monitor System, is developed. This is basically something similar to ShareOS, but just for compiling Fortran code. In 1960, we really reached the point of no return. IBM announces their own, very real this time, first-party operating system developed inside IBM for all IBM hardware, called IBSYS. It was meant as a replacement for older systems like

SOS and FMS. The new system was built for the batch, like all before it, and would grace machine rooms well into the future. And thus, we reach the end. Batch processing is one of those ghost stories that, luckily, most of us don't have to experience. While batching still exists in certain nooks and crannies, cough cough, IBM still uses it sometimes, Very few folk will actually come face to face with this monster. But we have to ask, is the batch all that evil? This is a surprisingly complicated question. I know I really wanted this to just be spooky, but there's nuance to this. We have all these stories of the horrors of batch processing, and the curses of programmers. It's clear that folk did not like the batch, and switched to more interactive modes of computing as soon as possible. There's this whole parallel evolution

of timesharing that's occurring just as batch processing is really digging itself in. By 1961, CTSS, the first operational large-scale timesharing system, is up and running at MIT. For Kurtz's part, the Dartmouth timesharing system is operational by 1964. Programmers were very eager to find a better way to use computers, But, despite newer technologies, batch processing would remain in use for quite some time. This is a case where we have contemporary evidence that people did not enjoy this technology. So, how could it be a good thing?

Simply put, batch processing was what allowed these complaining programmers to use a computer in the first place. It was the most simple, practical way to share a computer between many people. In an era where computers were not only scarce, but also pretty limited, batching may have been the only way to go. It sucked, it was annoying, and it was hard to deal with. But it may have been the best option, or rather, the least bad option. Without batching, I can imagine a generation of horror stories about setting

up and tearing down a mainframe by hand, or even just wishing maybe one day you too would get to see a computer. Thanks for listening to Advent of Computing. I'll be back in two weeks time with another piece of computing's past. If you like the show, there are a few ways you can support it. If you know someone else to be interested in the history of computing, then please take a minute to share the podcast with them. You can also rate and review the show on Apple Podcasts. If you want to support the show directly, then you can buy merch from my online merch store or sign up as a patron on Patreon. Patrons get early access to episodes, polls for the direction of the show, and bonus content. You can find links to everything on my website, adventofcomputing.com.

If you have any comments or suggestions for a future episode, then go ahead and shoot me a tweet. I'm at adventofcomp on Twitter. And as always, have a great rest of your day.