Advent of Computing - Episode 99 - The Berkeley Software Distribution

Episode Date: January 9, 2023

UNIX is a big deal. It's one of the most influential programs in history. Most operating systems that we use today can trace their lineage back to UNIX. The only notable exception at this point is Windows. But all these new-fangled operating systems aren't blood relatives of UNIX, they are all derivatives. Second cousins, if you will. So how did we get from UNIX into a diverse field of UNIX-like things? It all starts with a little project at UC Berkeley.   Selected Sources:   https://archive.computerhistory.org/resources/access/text/2022/06/102743073-05-01-acc.pdf - Oral History of Bill Joy   https://archive.org/details/aquartercenturyofunixpeterh.salus_201910/page/n157/mode/2up?view=theater - A Quarter Century of UNIX

There are few pieces of software as influential as Unix. This operating system was developed at AT&T's Bell Labs in the late 1960s, and from there it's really spread to fill almost every corner of the digital world. We even have a nice canonical date for the start of this new Unix era. Within the operating system itself, time is defined as the number of seconds since 1970. This is just called the Unix Epoch, sometimes, confusingly enough, shortened to Epoch Time. Unix is this wellspring from which much of the digital landscape was birthed. If you are connected to the internet in any capacity, then you're actually connected to

a machine out there somewhere running a Unix-derived operating system. If you have a smartphone, either Apple or Android, then you're running some form of Unix-related operating system right in your pocket. Even tiny things like microcontrollers can be made to run Unix-like software. It really is everywhere. But notice something in my last carefully worded statement. I never said any of those things are running Unix. At least not Unix proper, not Unix with a big capital U. This is something that I find fascinating. You see, no one really uses the Unix that was written at Bell Labs, at least not anymore. We use things like Linux, an operating system developed to be compatible with Unix, or BSD, another operating system

derived from earlier versions of Unix. Both of those things, Linux and BSD, were written by people outside of Bell Labs. You can plot this whole family tree of systems connected somewhat to Bell Labs. Many of those systems are still in use today, but not Unix itself. AT&T did make a lot of money here, but they didn't have the last laugh. Unix would get away from them. Years later, in the early 1980s, we would see the same thing happen at IBM. Big Blue's PC was pretty darn successful in its own right, but compatible machines, the clone market, that would really blow things up to the stratosphere. Unix initially came out as source code, so anyone could muck about in it,

as long as you could read C. AT&T didn't push Unix far enough or fast enough, so other parties decided to pick up the slack. This gives us a kind of funny situation where Unix-like things have won. They own the market. But without AT&T's current involvement, Unix would outpace its first home, or, to look at it another way, it became too big for AT&T to handle alone. How did this process start?

Well, it all comes down to a group of talented programmers at UC Berkeley. Welcome back to Advent of Computing. I'm your host, Sean Haas, and this is episode 99, the Berkeley Software Distribution. Now, I've been in a Unix-y mood lately. To be fair, I'm usually in some flavor of that mood. I'm an avid Linux user, both at work and at home. I think the only thing I use Windows for at this point is playing a few specific games. One of those is Skyrim, I will admit. I'm also one of those evangelical Linux aficionados. I think that everyone should use Linux. There are very few reasons not to at this point. The problem,

though, is the learning curve. Linux and Unix derivatives in general can be really hard to get into. Outside of all the technical stuff, you just have to come to grips with the entire environment surrounding Linux and Unix. Now, don't get me wrong. Linux is a great operating system. It's the most customizable thing on the planet. But therein lies the issue. There are hundreds upon hundreds of different distributions of Linux that you have to choose from.

Let me try and explain this as succinctly as possible. Linux is actually just a kernel. That's the core part of an operating system. The kernel hosts programs, the code that you actually interface with, the stuff you run. The thing is, the Linux kernel doesn't come prepackaged with software. You could try and sort all that stuff out yourself, but that's a fool's errand. I do not recommend it.

It's a massive headache, to say the least. So groups are formed that bundle up different software with the kernel. Those bundles are called distributions. They can vary widely, but are all compatible, at least for the most part. This can be really confusing to a newcomer. I used to use CentOS at work, but the project has since been discontinued. At home, I use Debian, but I've also been known to play around with a little Arch and some Mint. I have friends that like Manjaro, and I've even met people that use Gasp, Gentoo. That all should sound like nonsense, unless you're an old hand at Linux, it's an almost impenetrable barrier

to entry. You can't just go online and download Linux. You have to figure out which distro you want to run. That can be a tough choice. So why are things like this? There are a few possible reasons, and I personally think it's a legacy problem. You see, Linux is based on this older operating system called Unix. And, well, Unix had its own distribution confusion.

You have your old-school System 6 Unix, or System 7, or System 5. They're all slightly different. Or you could go commercial and get an SCO Unix, or System 7 or System 5. They're all slightly different. Or you could go commercial and get an SCO Unix license. If you're more into open source, then you can use BSD, or FreeBSD, or OpenBSD, or NetBSD, or DragonflyBSD. If you own a PC, you can use Xenix. If you own a Mac of a certain vintage, you could use AUX. Of a different vintage, you could use Xenix. If you own a Mac of a certain vintage, you could use AUX. Of a different vintage, you could use Xenix also. It's a mess. It's a tangle that's really hard to understand unless you already know what you need to know. There's just so many Unix-derived and Unix-like things that we even have a word that encompasses all of them. It's asterisk nix, or nixes,

since, you know, most of these things end in n-i-x. Some also refer to them as unices, or unixes. The point is, we're looking at a really wide field of options, a field so wide and so dense that the outsider is effectively kept at bay. So how did this all start? Why are there so many different Unixes and Unix-related operating systems? It all goes back to the complications around early Unix itself, and a project known as BSD, the Berkeley Software Distribution. This would be the first major distribution of Unix that was made by someone outside Bell Labs.

If we're looking for a root of the distribution confusion, then this is the best place to start. Today, we're going to be tracing out the roots, the early origins of BSD. To be 100% clear, this episode is going to have a limited scope. I've been avoiding covering this period of Unix for a while. This era, or at least what we're going to be butting up against, is called the Unix Wars period. That's from roughly the 80s into the 90s, where different versions of Unix all competed for commercial market share. It's a really complicated web and definitely a daunting task to cover, so this time we're only looking at the earliest phases of BSD, not necessarily

its rise in competition with other Unices. That's something that we might look into later, and very slowly. So we're going to be talking about how BSD got started, what makes a distribution of Unix special in the first place, and how BSD would shape the early Unix community. All of these are crucial questions to answer if we want to understand the Nix world of today. And really, when it comes to the internet or computing in any professional sense, the Nixes are where it's at. But before we dive into the episode proper, I need to give my usual plug for Notes on Computer History. That's the non-academic journal that I've been working to get off the ground. We've come a long way. We have a total of five articles right now that are through

editing. I'm trying to get five more, though. So if you're interested in writing about computer history, then please get in touch with me. And you don't need to have a background in academia or technical writing or even history. If you're interested in doing this, then I want Notes on Computer History to be a place for you to write. So if that sounds like you, and if you're listening, that is you. Welcome. Please reach out. You can go to history.computer to get information on how to write and submit an article.

With that out of the way, let's get into the show proper. Here's something we need to understand from the start. Unix was born in New Jersey. It was initially a small operating system developed for the PDP-7 by Ken Thompson and Dennis Ritchie. system developed for the PDP-7 by Ken Thompson and Dennis Ritchie. At the time, circa 1969 and before, the two were working at Bell Labs. Bell had just dropped out of the Multics project. That was a big multi-agency effort to create a super operating system. Things didn't work out for a variety of reasons, so Bell and AT&T decided to go their own way. That left programmers like Thompson and Ritchie kind of stuck in the lurch. They liked programming on Multics, but now they didn't have access to the operating system.

Over the course of a few months, the duo hacked and patched their way into a small replacement. And thus, Unix was born. The name, of course, being a pun. If Multics was big and multiple, well, if you only had one of those, wouldn't that be Unix? Anyway, that's a big story to pack into a short paragraph. I just want to give us a little context for two points. First, the lab that Thompson and Ritchie were working in was located in New Jersey. That's on the eastern side of the United States. And second, Unix was, from inception, an operating system designed by and for programmers.

This was a digital playground for hackers and tech heads. It's little surprise that as Unix grew in popularity, it would become the backdrop for whole swaths of computer history, especially hacker culture. But for such an important touchstone, Unix was a bit of an ugly child. It started life in the bowels of Bell Labs. It stayed there for years, fermenting into a more mature system. The first time Unix was ever even uttered outside of the lab was the 1973 Symposium on Operating System Principles, a regularly occurring ACM event. This was still in the realm of research, but at least it was a chance to get out from under the thumb of AT&T. The first outing was a talk given by Thompson and Ritchie, backed up with a paper titled The Unix Timesharing System. If we were chronicling the

spread of a new virus, this would be the epicenter. Dr. Bob Fabry was in attendance, a professor of computer science at the University of California at Berkeley. Fabry was already steeped in the world of operating systems. As a student, he had contributed to CTSS, the Compatible Time Sharing System. That was one of the first of the so-called time-sharing operating systems. These systems allowed multiple users to share computer time. They could devote a single computer to multiple jobs. So Fabry knew what a big deal Unix could be. These timesharing systems were big, complicated pieces of code. If there was a new one on the block, and a pretty simple one, well, that would be a big deal. He decided he had to check out this Unix thing for himself.

And thus, the woes began. Unix had run through a number of computers over at Bell, but the current version had only one compatible platform, the PDP-11. This was a pretty big machine made by Digital Electronics Corp, aka DEC. I can't find the exact price in 73 for the exact loadout you'd need, but we're looking at something that costs over $10,000, that inflates up to $70,000 modern US dollars. So while this was cheaper than really big hardware, it's still pretty darn expensive. That's like a nice new car. So Fabry had to do some wrangling to get his hands on one of these machines. The final arrangement was that the computer science, mathematics, and statistics departments would pool resources and get a brand new PDP-11.

The compromise, though, was that these departments would share computer time. Fabry, at the time, was the only one interested in Unix on campus. So we get this weird situation where there were 8 hours each day dedicated to Unix, and then the rest of the time was another operating system called RTS that the math nerds wanted to use. That was the hardware hurdle crossed, but there was still the matter of software. You know, Unix itself, the whole point of this exercise. Fabry couldn't just go down to his local CompUSA and buy a big box version of Unix. There was actually no nice mechanism for distribution of this

kind of software. If Fabry wanted something from DEC, that would be easy. Since the university now owned a DEC computer, they had a contractual relationship with the manufacturer. DEC, for their part, had all this fancy software already tailor-made for their own hardware. Just make a call and a technician would be dispatched. Easy. But with Unix, things were different. To start with, Bell actually had a legal mandate that Unix had to be distributed. You see, AT&T had antitrust issues. Bell Telephone Company and, by extension, their labs, had this problem with creating monopolies, which got them sued and trust-busted quite a few times over the years. One of these court cases was brought by the DOJ in 1956. It forced Bell to

operate strictly within the bounds of telephony and other communications technology. They could only profit from their main business. So what should they do about Unix? Bell was required to license any non-telephone stuff to anyone who asked, and these licenses had to be on an at-cost basis. This meant that anyone who asked for Unix could get some Unix, and pretty cheap to boot. AT&T did not like this arrangement, but it suited Thompson and Ritchie just fine. We'll find that they were pretty anti-establishment type of folk. Distribution worked something like this. Fabry or another interested party would send a letter to Bell Labs

or call up someone saying, hey, we want to license Unix. They would sign off on some paperwork, and then the request would get passed to Ken Thompson. He would run off a copy of the Unix source code onto a tape and include some documentation and a note. He'd also sign off those notes, Love, Ken. So each version of Unix was actually made with a little extra love. The tape would then be sent off in the mail with the new licensee only paying for media and postage.

Bell would have loved to charge more, but legally, Unix had to be free, or at least close to it. That's the path that brought Unix to Berkeley, out to the West Coast. At this point, there were only a handful of Unix installations outside of Bell Labs, and Fabry now had access to one of them. Or he would after a lot of setup. The operating system still had to be compiled, then installed on the PDP-11's hard disk, and probably tweaked a little bit so it would actually run. Early versions of Unix were notoriously unstable, so things wouldn't have been smooth going. But soon enough, Fabry was face to face with the wave of the future. What was done with this new fancy Unix thing? Well, nothing too flashy,

actually. Remember, this is still very early into adoption. There are probably dozens of users, if we're being generous. So from the beginning, the major use of Unix was software development. Fabry ended up using Unix as part of his computer science curriculum. It made for a wonderful, if slightly unstable, programming environment for his students. One of the first big projects that the Berkeley crew undertook was the development of a new database. That should give you a flavor of the kind of stuff going on here. We're dealing with big, serious software written on serious computers. Tools and programs meant to be used by other programmers, not necessarily anything a non-programmer would care about. Over the next few years, things would go pretty well for the comp sci department. This new database, Ingress, was showing promise.

The department was growing, and the odd 8-hour Unix days just didn't cut it anymore. Money was scrounged and saved, and in 1975, the department bought its very own PDP-11. This time, it would be all Unix, 24 hours a day. I'd say 7 days a week, but we're not entirely sure how much of that time was real uptime. This coincided with another big event. Ken Thompson was taking a sabbatical from Bell Labs to come out and teach at UC Berkeley. Thompson had actually earned his master's degree at the school. I'm sure his alma mater was pleased to welcome back a successful graduate, if only for a short time. Thompson also came along with a new version of Unix, V6.

This is kind of the golden age of the operating system, at least during its research phase. V6 would be the first version to see real widespread use. It would be enshrined in Lyon's Commentaries, an operating systems textbook that used Unix as a case study. Its source code would be spread far and wide, despite Bell's attempt to clamp down on licensing. Thompson would actually help install Unix v6 on the comp sci department's new computer. I like to think of this as one of the important acts that bridged the Unix world together. Knowledge, code, and even personnel were leaking out of Bell's secluded labs. That would prove crucial to Unix's growth and survival. System installations weren't the

only thing that Thompson worked on during his sabbatical. That would be a very boring year off. He also started a new project while at Berkeley, a Pascal compiler for Unix. This was the kind of thing that made for a really good student trap. Thompson would only spend a year or so working on the compiler, which left a project in a near-complete state. There was still room for improvement, but a lot of the groundwork had been laid out. A few grad students wound up taking over the Pascal Project after Thompson went back to Bell Labs. One of those students was named Bill Joy. He had actually just arrived at Berkeley himself.

Joy had been shopping around for grad schools and ended up choosing Berkeley because, well, it was among the cheapest options for him. That's something I think a lot of us can relate to. At this new school, Joy would meet a new type of operating system. From his oral history with the Computer History Museum, quote, Berkeley had no large computer systems. When I arrived, it only had a 16-bit computer, which is a relatively small computer, a PDP-11, as opposed to the PDP-10s, which were much larger computers that all the other universities had. This little PDP-11 ran Unix. End quote. This is a point that's easy to lose

in the shuffle. The PDP-11 is remembered as a really iconic computer, but at the time, computer, but at the time, it was on the smaller side of things. The more powerful machine in DeX lineup, the PDP-10, wasn't just a bigger version of the 11. It was a totally different computer, incompatible with the smaller machines that outfits like UC Berkeley could afford. Unix was filling this really particular niche where it fit on these smaller machines like the 11. In the coming years, Joy and his colleagues would whip Thompson's Pascal compiler into shape. They made it more stable, gave it better error checking in messages, and just generally made it more useful to program with. To do so, however, proved to be a bit of a roundabout process. These were still the early days of Unix.

It was missing many key tools. One notable absence was a passable text editor. The editor that shipped with Unix, at least the one provided by Bell Labs, was called ED. This was probably where programmers would spend most of their time. The shell was Unix's main user interface, a simple interactive text prompt. That was good for running programs and shuttling around data, but you couldn't program, you couldn't write code with the shell alone. You needed a way to edit simple text files. Ed was really the only choice, but it was not a good one. In fact, that may be a bit of

an understatement. Ed seems to be closer to evil than really anything. It's a line editor, meaning that you have to edit your file one line at a time. You also have to issue these cryptic commands to facilitate editing. If that wasn't annoying enough, Ed also doesn't use any display or any prompt or any meaningful feedback for the user. You just run Ed and you're greeted with nothing. You have to be an adept user to do anything at all. If ED was so bad, then why was it in use at all? Well, that comes down to tradition, stubbornness, and hardware limitations. Unix was initially developed on a teletype terminal, one of those fun hard copy computer terminals that works more like a souped-up electric typewriter. These terminals

give the programmer a very narrow view into the computer. You get one character at a time, one line at a time. With this interface, a line editor is really the best you can hope for. The tradition part is a fun twisted mesh. When Thompson was a student at Berkeley, there was this text editor in use called QED. It was also a line editor, and it appears that Thompson lifted many of the features of Ed directly from QED. Thompson had learned to program this way, and he appeared to just like it for some reason. From what I've read, Richie was of a similar mind, so the core Unix developers never really cared to implement a better text editor. By 1975, there were straight up better options. Glass teletypes, terminals that used CRT displays, if you can believe it,

were just starting to hit the market. This gave programmers a more 2D window into the machine. You could actually display multiple lines of text, and some of these terminals even made clever use of this new real estate. We're talking cursors that moved in two axes, the abilities to update specific locations on the screen even. This would open up a world of possibilities, if you had the hardware. Bell Labs did not. Apparently, they wouldn't get onto the new bandwagon for years. All these factors contributed to keep the official Unix text editor stuck in, essentially, the Stone Age. You basically can't use ED to get anything done. I know this for a fact because, horror of horrors, it's still in stock installations of modern Linux. If you use Linux, then you're one step away from using Ed.

I don't recommend it. It seems like the only people who liked this text editor worked at Bell Labs. editor worked at Bell Labs. Every other Unix user wanted something better. One of the earliest passes at this task took place in Queen Mary College in London. George Cullerys, a student with access to one of the few Unix installs in the world, created this little editor called EM. It stood for Editor for Mortals. Given how poorly ED worked for most humans, this seems like a fair title. The program was initially developed in 1973 as an extension to ED. One of the nice side effects of the source-only distribution of Unix was that any user had easy access to all of the Unix source code, not just the kernel. Caloris didn't have to write M from the ground up. He could just grab the code for Ed and add his own tweaks. The big important one of these tweaks was implementing cursor keys. M didn't display a whole file at once,

just a single line, so in practice, this was little better than the Bell original, but the grain of something was there. Just the fact that you could actually edit a line of text with instant feedback, that you could move left and right, that was a big deal. As with all software of this era, M had to physically travel to reach new computers. In the summer of 1976, Caloris visited UC Berkeley, and he brought M along with him. This is where Bill Joy would be exposed to the idea of a more interactive text editor. In A Quarter Century of Unix, the big main source that I'm using for this episode, Peter H. Salas gives a great summation of this period of Unix. He describes this era as one of powerful exchanges of ideas and source code.

This exchange would lead to something that Bell couldn't handle. There was no way that patent and licensing could control this, and the system got better and more widely used all the time. End quote. I think this phenomenon goes even deeper. Early Unix didn't work that well on its own. It was honestly kind of garbage. Ed is a great example of this, but there were more fundamental issues. Unix wasn't stable code, not yet. It crashed, it hung, it was full of bugs, and Bell was trying all the time to turn it into a product somehow. Around 75, Thompson & Co. created this big tape called the 50 Bugs Tape. It was 50 critical bug fixes for Unix. Bell refused to distribute it, so the Unix community stepped in. The tape was leaked and passed around clandestinely.

According to folklore, the tape was left in some type of dead drop outside Bell's offices in New Jersey. Someone just picked it up off the street and decided to pass it around. Unix was flawed from the start, but there was the kernel of something great there. A big part of that greatness was the community that sprung up around Unix. It was cheap, it could run on cheap machines, and it was open source before that word had much cachet. Its flaws helped in this regard. Unix called out to hackers and programmers, it was asking to be fixed, asking for them to tinker with it. Clandestine operations like the 50 Bugs tape helped to draw the community closer together. They were bonded

in secrecy to an extent. And they were building their own little world. So when Joy saw this visitor from London using his own custom text editor that was built on top of Ed, well, there was only one thing that he could do. Joy ran with the modified source code, threw in his own flair, and made X, the Extended Editor. It operated in a similar fashion to Ed and M, except it had this thing called open mode, or sometimes visual mode. In this mode, X expanded to fill the whole screen of a glass terminal. Fill every line of glass with text. You could move right, you could move left, but you could also move up and down. It was the first full-screen editor for Unix. After wrestling with Ed, it seems like everyone liked this new text editor. By 1977, UC Berkeley had a good pile of in-house Unix software. X would be used to finish whipping Thompson's Pascal

Piler into shape. It turned into something that students could use and enjoy. X, of course, had thrown Unix into two dimensions, if only in text mode. But remember, we aren't in isolation anymore. UC Berkeley was just one node in the larger Unix scene. The software couldn't just be hoarded. Joy even benefited from Unix software from another campus. So Joy took the next logical step in the story. He started compiling a distribution. This would eventually be called 1BSD, or the First Berkeley Software Distribution. It wasn't a full Unix system yet, rather just a pile of software developed and used at the Berkeley Computer Science Department. The cornerstones

were, of course, Pascal and X. That said, there was actually a lot of software in one BSD. The package is pretty well preserved, so we can take a look at it today. The package is pretty well preserved, so we can take a look at it today. The two other programs we find are a mysterious thing called A-Shell and a Star Trek game. A-Shell appears to be some sort of custom command shell. We only have a binary file, so I have no clue what it actually is. The other program, Trek, is a pretty well-known computer game. It was written in 73 and Basic, but was pretty quickly and easily ported to every machine imaginable. What can I say? Computer nerds tend to like Star Trek.

This is Berkeley's own version, written by Eric Allman. Now, 1BSD would be distributed in the same way as Unix in this period. If you wanted to partake of some quality software, you'd call up Bill Joy and make arrangements. For just $50, Joy would run off a tape and send it out to you. No reports on if those tapes were signed Love, Bill, but one can hope. The first tapes would start shipping in March of 1978. This was the beginning of BSD. It was a small start. About 30 tapes were actually sold, but hey, it proved that there was interest in this new Unix software. It was enough to keep Joy going. So let's take a quick diversion here to talk about distribution. Specifically, was one BSD

really that remarkable? Looking from our vantage point, it's easy to get a warped view of earlier events. Today, BSD is one of the more popular versions of Unix. It's a force in the server world, and it's even spawned its own derivatives. One of those is macOS, oddly enough. But in 1978, was it special yet? This type of at-cost software distribution had been going on for years. Bell Labs didn't start some trend, and neither did BSD. One example that I can think of is Share, an IBM user group that cheaply distributed software and also collaborated on larger projects. There are equivalent

organizations in the Unix scene. The most well-known is Usenix, a Unix user group that was formed back in 1975, so it predates 1BSD. The difference here is that Share was more explicitly about software sharing, while Usenix seems to be more about conferences. However, there was an outgrowth of Usenix called the Software Tools User Group, or STUG. This group was specifically writing and distributing Unix-like software as portable source code. The idea was to make the Unix philosophy available to any operating system. This might also be a neat topic for a future episode. Anyway, the point here is that there were people distributing third-party Unix software

and doing it either cheap or free. 1BSD falls into a growing tradition. It's an early addition, but not the origin of the trend. This would have only helped Joy spread around Berkeley's software. He didn't have to blaze a new trail. The Unix community was already ready for this sort of thing. It was just one in a long line of new packages of programs. If things had stopped at 1BSD, then this would have just been remembered as a footnote. But Joy would have the last laugh. Well, laughs, I guess. There are a number of other versions of BSD to come.

The first release would be followed up by 2BSD. The first release would be followed up by 2BSD. This is where things get more spicy, where we start to see familiar software cropping up. One interesting aspect of all Unisees is that they're very customizable. So the Nix user experience isn't defined so much by the core parts of the operating system. Most users never get very deep into the digital waters. There are two things that actually matter to run-of-the-mill Unix lovers like myself. The shell and the vast array of available programs that you can run from the shell.

Today, we still use a lot of programs that were developed at Bell Labs. None of the code is the same, everything's been rewritten a number of times at this point, but we still have the fundamentals. We have the same names that do the same things. LS has always been the command used to list files in a directory, since the don of Unix, for instance. Mount is used to connect to storage media. DF shows you how much free space is on those disks that you've mounted. The point I'm getting to is that if a modern Linux user was time-warped back to 1971,

they could probably get a job at Bell Labs using a very early version of Research Unix. That said, there are notable differences. The first big one is Vi, the visual text editor. Now, much ink has been spilled over which text editor is best. Some say it's MIT's Emacs. Some say it's Vi. And there are some dark horse contenders that occasionally come up in the conversation. Whatever side you fall on, Vi is undoubtedly a staple of any Unix-like environment. It's simple, it's fast, it's lightweight, and it's loved by programmers the world over, even if you may not like it. It's so integral to the experience that it's really easy to think that it's always been part of Unix, but as we've seen, that's not the case.

Vi is actually pure BSD. We've already met the editor. You see, here's something that I never knew until recently. Vi is actually just the visual mode of Joy's X editor. Over the last few years, X had seen increased use at Berkeley, and something interesting had happened. There was this very specific issue with editors like MX and later Vi. To do the interactive cursor thing, they had to use this thing called Unix's RAW mode. That terminal mode gave the programmer more control over, well, the terminal, but it came at a cost. The program required more CPU time to process keystrokes. Each button press would actually require the processor to go do something. It's just a higher degree of interactivity. In a normal terminal session, as you're typing stuff in, the terminal actually handles, you know, typing out the text you hit. It only

talks to the computer when you hit enter. So it might be running a bigger job every time you send something in, but you're sending stuff in less frequently. In raw mode, every button press means the computer has to wake up and do something. Unix ran on shared hardware with multiple users, thing. Unix ran on shared hardware with multiple users, sometimes many multiple users. So at first, the Berkeley Computer Science Department was very concerned about the use of M. As Caloris recalled, the system support person said something like, that's very nice, but if we made it available to all of our users, the overheads associated with running in raw mode would swamp the CPU. End quote. That would be a really big problem. M would find its way onto Berkeley's Unix systems. It turned out that no amount of concern could stop it.

No one really wanted to use a line editor. Thompson and Ritchie were outliers, the immortals as it were. There would be performance degradation, but it turned out that no one cared. That's kind of the funny thing here. Performance is really nice and all, but it's not worth anything if the high-performance program is garbage. So the editor for Mortals really came into its own. X improved upon its features and supplanted M. The curious thing about X was its visual mode. Berkeley had already made the jump into ROM mode. It had been decided that usability really trumped performance. So why not use ROM

mode to its fullest extent? Once you taste a more visual editor, it turns out there's no reason to ever go back to line editing. Thus, we reach Vi, at least in a manner of speaking. Initially, Vi was the same as X. Typing Vi at the command line would just load X straight into its visual mode. There's something I find really neat here, and really Unix. If you're keeping track, there's actually a direct connection between the evil Ed and the lovable Vi. Despite all the changes, Vi was still based off work done at Bell Labs during the very first days of Unix. So even by 1976, there's this feeling of history and lineage that's lurking inside the software. It's a collaborative system, one where each program has its own pedigree, and each programmer

is really standing on the shoulder of their predecessors. I think that goes a really long way towards building a community. That's something that a lot of software lacks. The other big addition to 2BSD was a program called the C-Shell, or CSH. This was another of Joy's pet projects. In Unix, a shell is the interactive command line that makes up the user interface. At least, a shell is the default user interface. We have some pretty fancy graphical Unix stuff today, but the shell, that's where it's at. One neat thing about the shell is it's actually just a normal program. It's not a core component of the Unix operating system. It's actually no

different than something like Vi. This means that you don't have to use the default shell. You can use something different if you feel like it. You can use any program you want. You can use something different if you feel like it. You can use any program you want. The de facto shell for Unix was SH. That was what it shipped with. It's retroactively been termed the Thompson shell. It's simple, as is the case with most of this early Unix stuff.

The C shell and the programs that were packed with it improved on SH in a number of ways. Most notably, at least in my opinion, was history. Most Nix shells will keep track of every command you run. This is another one of those ubiquitous features that didn't actually stem from Bell. History, at least in the shell sense, was added to the mix at Berkeley. I'm not going to go down the whole list of improvements made in CSH, but I do want to drop one more biggie. Job control. This, I think, is really cool, and it's something that I use all the time.

CSH lets users start jobs in the background, or send a job to the background. This leaves the process running somewhere off on the computer, but it drops the user back to CSH. In other words, it's a way to multitask without something like multiple windows. From any terminal, be that glass or paper, a Unix user was able to multitask. This is another core feature of Unix-like operating systems that came from Berkeley, and that kind of blows me away. 2BSD was released in the summer of 1979. The four core programs, Pascal, X, Vi, and CSH were backed up by a pile of bug fixes and improvements to already existing Unix utilities. This made 2BSD a much more useful release than version 1. It's been reported that around 100 copies of the tape shipped, making it a pretty big success all things considered. It's hard to

tell how many machines were actually running Unix at this point, but two BSD would have been reaching a good portion of them. So where was there to go from here? Well, three BSD of course, you can always just keep increasing that number. But the next iteration wouldn't just be a bundle of software, oh no. This would be the first release of BSD that was a fully-fledged Unix distribution. The next step required a bit of a push which came in the form of a new computer on campus, the DECVAX. The VAX, short for Virtual Address Extension, is a computer series that's held my fascination for years. These are 32-bit machines that are somewhat compatible with the earlier PDP-11 architecture. There are two main differences, though.

More bits means the VAX could address much more memory. And then there's the whole virtual address extension thing. Virtual addressing or virtual memory really makes up a big topic of discussion. I don't want to get into the whole thing since it quickly gets very technical. The gist is that virtual memory allows a computer to pull tricks with how it presents memory to programs. You can, for example, present what looks like a continuous chunk of memory from all over the computer. This could include data on disk drives, physical memory in banks of RAM, or even memory-mapped I.O. devices. For these tricks to work, you need some backup from hardware. That's the only way it can have good performance, really.

The VAX architecture was designed specifically for this task. Why would anyone want to play memory tricks like this? I mean, it sounds kind of complicated, right? Well, it all comes down to multitasking. Kind of complicated, right? Well, it all comes down to multitasking. Virtual memory lets you more effectively isolate a program from the rest of the system, which can prevent a rogue program from crashing your entire computer.

That's a very good feature to have. It also gives the operating system more flexibility in how it can run user programs. The trade-off is complexity. The operating system has to know how to deal with virtual memory hardware. Bell's version of Unix for the VAX, Unix 32v, well, it didn't take advantage of what the VAX could do. This actually ended up lining up with other issues at large. The whole point of a VAX was that it had virtual memory, that it was a machine specifically designed for fancy multitasking tricks. Anytime you have a new feature like this, it expands the boundary of what's possible.

It makes new kinds of software feasible and research easier to undertake. So labs and tech heads the world over wanted to use it. They wanted to make use of these new VAX features. Unix was really nice and all, but it couldn't use a VAX to its fullest extent. That said, Unix offered something that other operating systems didn't. It was portable. At this point, Unix was written in C, a language designed to be platform agnostic. With a minimal amount of work, Unix software could run on anything.

At least as long as that anything could run Unix. Portability was one of those features that programmers wanted to make use of. This was exacerbated, at least on the DeX side of things, by some bad choices. Once again, from Joy's oral history. Quote, As I mentioned before, the PDP-10s all ran different operating systems. So if DARPA wanted to fund some research, say, at Stanford, and there was some other research at Carnegie, they couldn't share software because they were written in different programming languages for different operating systems. DARPA wanted to have a common operating system it could use across a lot of different programs at different universities.

They wanted it, in fact, to be hardware-independent, too. They could imagine carrying that software into the future. End quote. The whole point of an operating system is that it manages hardware resources for user programs. Each operating system tends to do this in its own way. A Mac program doesn't work on Windows, even if both operating systems are running on the same hardware. doesn't work on Windows, even if both operating systems are running on the same hardware. Back in the 70s, there was the added complication of supported programming languages.

Something like MIT's ITS operating system really wanted you to use Lisp. IBM operating systems would prefer that you used Fortran. And Bell wanted you to use C. Beyond just incompatibility, this diverse field made it hard to manage collaborative projects. DARPA was just one organization that was fed up. Joy didn't have a connection to the Feds quite yet, but this malcontent was brewing in the background, so I want to take note of it. Nunex could solve a lot of these compatibility issues. If virtual memory could be added to the mix, that would make Unix all the more promising. The best part here is that there was no barrier to modifying Unix itself. Joy and his colleagues already had the Unix source code just, you know, hanging out on their hard disks. Every Unix user

the world over had access to this code. Up to this point, Berkeley had made some patches here and there, some new utility software, some bug fixes, but never made any wide-reaching changes to Unix. 32v gave them an interesting opportunity. This would lead directly into 3BSD, a new type of distribution. The VAX already had its own fancy operating system. It was called VMS, the Virtual Memory System. Can you tell what DexMain's selling point was? VMS was meant to be THE operating system for the VAX series of machines. That's a really nice idea and all, it certainly made things easier on DEC. The problem was that this isolated VAXes from the rest of the world. Nothing else ran VMS, just this one new line of computers. That runs pretty counter

to the whole portability thing that nerds like DARPA wanted. At the same time, VMS did the whole V thing. It was programmed to make especially good use of the VAX's virtual memory hardware. 32V, although interoperable with other Unix systems, was missing the VAX's main selling point. So what if you could combine these two operating systems? Why not make a version of Unix that supported virtual memory? Well, Bell just didn't want to. Thus, the crew at Berkeley took up the task. We're no longer in the realm of simply making new software or making small tweaks, but now we're pretty deep into the dungeons of Unix. Memory management is one of the core features of an operating system, and I really do mean core here. There are only a handful of resources that a kernel really needs to manage.

You have storage, processor time, input-output hardware, and memory. Almost any operation you run will require some amount of memory. You want to execute a program? Well, you need to ask the kernel to give you some space in RAM to load your code. You want to make a list or an array? Guess what? You need some memory for that. Need to even just load a file? Believe it or not, memory is involved. Adding virtual memory support to Unix meant that the Berkeley team would need to make fundamental changes to the operating

system. That was no small feat. The first VAX would hit campus in 1978, and it took a team of programmers to get this new memory manager in shape. And even then, the kernel wasn't ready until 1979. That's a lot of work. Virtual memory alone would have been a major undertaking. But there was more going on here. Before we continue, I do want to make a little side trip to note something I ran into in my research. I've seen some programmers recall that 3BSD included improved networking support, specifically for things like TCP IP. substantiate that. It shows up in some quotes used in A Quarter Century of Unix. I suspect that these are either misrememberings or just poorly worded statements. The actual release notes for 3BSD don't contain anything about networking, so I think the kernel modification stopped at virtual memory. This is just something that I wanted to clear up since it got me sidetracked. So watch out for this confusing detail if you venture down this research route. Anyway, the real bulk of the code came in the form

of ports and utilities. Joy didn't want to regress to the codebase, so he and his co-conspirators actually set about porting the software from 2BSD over to the VAX. In most cases, that was easy. A lot of this codebase was written in C, the portable language of the future. Bell had already done the hard work of getting a C compiler up and running on the VAX. So, Vi for VAX was just a recompilation away. It's a single step. The same process would follow for TREC and CSH. One stumbling block was actually the Pascal interpreter, which was apparently written in PDP-11 assembly language. I think the compiler wasn't, but I guess the interpreter just was for some reason. That would take some elbow grease, but that program would also find its way over to the VACs.

There were also all the patches and tweaked utilities from 2BSD that made their way over to version 3. So in general, we're looking at just a much expanded version of the previous release. Then we come to the most fun programs that were bundled into 3BSD. Games. fun programs that were bundled into 3BSD. Games. Besides all the practical stuff, 3BSD included a whole slate of Unix games. We do get some of the usual suspects here. The aforementioned Trek is ever-present, as is a version of Colossal Cave Adventure compiled for Vax. Adventure was really everywhere in this time period. I just think it's neat to see it showing up even in BSD. In all, this new version of BSD would actually contain 14 games.

Something that surprised me here is that most of these games actually came out of Bell Labs. I didn't expect that at all. Despite Thompson and Ritchie seeming a little bit humorless, at least when it came to programming, their operating system included a healthy supply of games. This surprise leads me to the next big development going on at Berkeley. So let's make a quick rundown here. We're looking at a modified Unix kernel, text editor, shell, and utilities. There's a lot of Bell Labs code here, but the changes made at Berkeley are pretty extensive. We've hit a point where it's more than just new software. We're looking at an altered version of Unix itself. These software changes also called for changes in administrivia, specifically how BSD was distributed. Up to this point,

software had been distributed as just packs of binary and source code. You were intended to load that tape onto an existing Unix installation, unpack the tape's contents, and then start playing around with your new software. But that could get dicey with a whole new kernel. What happens if things mess up? What if your machine goes down in the middle of the swapover? What if you only have half the kernel copied over? That would kill your computer pretty well. From Joy, once again, and just to preface this little quote,

in his oral history, Joy talks about 3BSD and 4BSD in the same breath. 4BSD would see the introduction of more robust networking support. The fact that these two releases are often talked about together I think is why I was running into the whole network confusion earlier. I just want to point that out before we get confused anymore. Anyway, from Joy, quote, Then we took the tapes, which had just been application software for the PDP-11, and extended them to be full bootable VAX operating systems. Now you could get a Unix operating system with virtual memory and, ultimately, the internet and all this software from Berkeley, rather than getting it from Bell Labs or digital equipment.

We're now up to something a modern Linux user could recognize, sans the tape storage, of course. When you go to install Linux, you don't call up the Linux Foundation and ask for a source archive. Instead, you usually just get a bootable disk from some third party. I personally use this distribution called Debian, so I get my boot disks from debian.com. Starting in 1979, Unix nerds were able to use this distribution called BSD, so they got their boot tapes from this place called UC Berkeley. But why go this far? Why jump from a simple software pack to a fully bootable version of Unix? This is one of those

questions that Joy and his colleagues just don't really address. I know, this makes it an annoying question to answer. Kirk McKusick, one of the programmers on the project, even just says that Joy decided to move on to a full Unix distribution. Just like that. No preamble, no explanation. So, let's take this question from a different angle. What benefit did 3BSD gain from being a full, bootable distribution? One simple answer is that rolling a separate distribution helped to move Unix out of the hands of Bell Labs and AT&T. The mothership had been

getting a little litigious as the 70s wore on. AT&T was looking for a way to legally commercialize Unix, ways to extract more control over licensing of the software. This ran counter to what many Unix users actually wanted. We're talking about academics and hackers, folks not really known for their love of authority figures. 3BSD represented an alternative to the capital U Unix. It let users get further away from AT&T's meddling. Sure, there were still legal issues to be had here. 3BSD was only a derivative work, not an entirely new system. There were still licensing clauses to think about, but hey, at least there was some more wiggle room here. Up to this point, any Unix user had to get their code from AT&T,

which meant they could be thrown on a list of licensees. A 3BSD enthusiast could now get up to speed with Unix without ever coming into contact with good old Ma Bell. Another possible reason could be ease of installation. If you had the right model of VAX, a VAX 11-780, and the right disk drives, you could just install BSD without much extra effort. Transfer the tape image onto your disk, copy over some files, and it should work just fine. You now had Unix with virtual memory and a whole lot of other goodies. lot of other goodies. From looking through old documentation, it appears that installing 3BSD was just about as hard as installing Unix 32v itself. That's pretty nice. This was only possible because, of course, 3BSD was wrapped up as a full Unix distribution. If it had been source or binaries like the earlier BSDs, then, well, that installation

process would be a lot harder. You'd probably have to patch or recompile the Unix kernel. If that sounds intimidating, then good. It is intimidating. It can be a slow and nerve-wracking process. Once again, this puts you in, really, the guts of the machine. If you mess up, you're in for a world of hurt. This also assumes that you already have a working 32V installation,

that you've gone through the licensing and paperwork with AT&T. So, at least in theory, sending out 3BSD as a standalone version of Unix meant that Berkeley could help spread the good word to more people. Now, I've seen it at least claimed that 3BSD was only distributed to sites with existing Unix licenses. I haven't seen this backed up by primary sources. To be fair, few of these early licenses are really easy to find, so I'm not entirely sure what the truth is. I think this claim is more of a legal smokescreen than anything, though. Personally, I doubt the BSD staff would actually care if they were shipping software to an official Unix site. I mean, even

Thompson and Ritchie, who worked at Bell, had been subverting AT&T's legal department for years. It wouldn't surprise me at all if Joy was just a little bit loose with what he did around licensing. Now, this new distribution took off like wildfire. 3BSD would start shipping in late 79. It's hard to tell how many tapes were distributed. Years later, Joy estimated that, during its heyday, there were as many as 1,000 sites using these early versions of BSD. That included version 3. So we can at least assume 3BSD was well-received. The big break, at least what seems like the big break to me, came when the feds finally got involved. As we've previously discussed, DARPA had some research issues.

Unix was close to solving their portability problem, but 3BSD just did it better. Bob Fabry was instrumental in making this connection. He worked out a deal that helped 3BSD get into the hands of the federal government. From Joy once again, quote, We had a group that Bob got us a grant for, and I got then a job working for the university as a principal programmer to develop the operating systems, develop Unix more to meet the technical requirements that DARPA had for its programs, end quote. This federal funding led to the formation of the Computer

System Research Group within Berkeley. This was the new home for BSD, backed by U.S. Department of Defense spending, strangely enough. The main modifications that DARPA wanted had to do with the growing ARPANET. Up to this point, the Young Network was populated with a wide variety of machines, each running their own operating systems. We're talking TOPS-20, VMS, ITS, WAITS, ELF, the list goes on and they're all names that are relatively obscure today. All were roughly incompatible. In the case of the incompatible timesharing system, it was very incompatible. With some modifications, Unix could squash out all of them. It could solve all the problems DARPA had. That is to say, BSD could solve all these problems.

Funding and development would lead to 4BSD, but I want to save that for a later date. With 4BSD, we start to dive into the Unix wars themselves. That's something that will require its own very drawn-out coverage. Alright, that does it for our look at BSD. Like I said earlier, I do want to cover the Unix wars. This might become another one of those slow burn projects if I see enough interest in the topic. It's definitely a period I'm not super familiar with, and I know there are a lot of great resources on it. That aside, what can we say about early BSD? Personally,

I keep coming back to what Salus wrote in A Quarter Century of Unix, quoting yet again, There was no way that patent and licensing could control this, and the system got better and more widely used all the time. Unix, much like the IBM PC, would become a huge deal due to outside forces. The PC was cloned by third-party hardware vendors. That led to an explosion. Unix was adopted and improved by a very dedicated community of programmers.

That would lead to a similar meteoric rise, but for different reasons. As PC clones spread, the cost of a machine went down, so it was easier for consumers to join in. As Unix got better and better, it just became the de facto platform for both hackers and more serious suit-and-tie style programmers. Its portability was a huge part of that, but so was its expandability.

Thanks to the community, Unix didn't languish, it got better with age. These improvements were a community effort. That is, until commercialization entered the mix. But that's a story for once we are firmly into the heat of the Unix wars. How's that for a cliffhanger? Anyway, that's it for today, but before I go, I do want to lay out a warning. This was episode 99, which means next time we'll be at episode 100. That's a huge milestone for me, and I'm super excited for it. Now, episode 100

isn't going to be normal advent of Computing fair. I'm planning it as something of a special episode, a celebration. So prepare for something different. I think you'll still find it interesting. At least, that's my plan. Anyway, thanks for listening to Advent of Computing. As always, if you like the show, please share it with a friend. You can also rate and review on Apple Podcasts. If you want to support the show directly, then you can buy Advent of Computing merch or sign up as a patron on Patreon. Patrons get early access to episodes, polls for the direction of the show, and a couple of bonus episodes. I think there's about six now. You can find links to everything on my website, adventofcomputing.com. And if you'd like to shoot

me a message, then the best place to reach me, at least for right now, is on Twitter. You can find me there at adventofcomp. And as always, have a great rest of your day.