Command Line Heroes - Diving for Perl

Episode Date: August 6, 2019

Languages come and go. A few have the right stuff to rise to the top—and fewer stay there. Perl had a spectacular rise, a quiet slump, and has now found its place in the world of programming. Perl seemed destined to rule the web. Michael Stevenson and Mike Bursell describe how Perl’s design made it ideal for the early web. We hear from Conor Myhrvold about its motto: “There is more than one way to do it.” Elizabeth Mattijsen shares how—despite Perl’s strength—a long development cycle slowed Perl’s growth. And although it’s not the top web language anymore, John Siracusa points out that Perl lives on as a niche tool. If you want to dive deeper into the story of Perl, head on over to redhat.com/commandlineheroes. Guest John Siracusa also co-hosts three podcasts. Check out Accidental Tech Podcast, Reconcilable Differences, and Robot or Not?

Picture Larry Wall, the creator of the Perl language, stepping up to the mic at a conference in 1999. He's got his signature bushy mustache and swept over banks. And he's feeling good. His language has been gathering steam. It's easily surpassed COBOL, surpassed Visual Basic, and Python? Python's still a distant contender.

Fodder for a couple of Wall's jokes. Wall shows the crowd numbers from a Dice.com report. Pearl's future is very, very bright. And then... It wasn't. Fast forward two decades to the summer of 2018, and Dice.com names Perl one of the languages most likely to go extinct. What just happened? I'm Suranya Dbarik, and this is Command Line Heroes, an original podcast from Red Hat. This season is all about the power and promise of our programming languages. Last episode, we tracked JavaScript on its crazy rise to success as it hitched its fortunes to the web.

But not every language has a story of nonstop growth and success. Most languages emerge, play their part in a very particular ecosystem of the moment, and then start to recede when our programming lives call for other kinds of tools. This episode, we're taking a deep dive into the world of Perl. What caused its early success? And why did that success suddenly get turned upside down? The environment we live in,

our infrastructure, our hardware, all kinds of factors will dictate which languages thrive and which begin to shrink. And that's what's so fascinating about the story of Perl. So we know that Perl didn't take over the world.

But back in the 90s, nothing was that clear. Tim Berners-Lee released the World Wide Web in 1991, and it quickly created a whole new frontier of web-based development. It was anybody's guess which programming language was going to succeed in that new terrain. The web emerged at a time that there was a lot of sort of like waiting for something to happen. There was excitement. Michael Stevenson is an associate professor in media studies at the University of Amsterdam. He described that earlier web for us.

People had seen Usenet, and they'd seen the ARPANET. They'd seen America Online and a few others. But the internet's full development potential really hadn't been expressed until the web came along. Suddenly, you're pointing and clicking your way into this vast, connected world.

It was a revelation. You have to remember that 1993, when the web sort of peaked its head up, that was the same year that Wired Magazine started publishing. Before that, you had things like Mondo 2000, these magazines that really kind of made computers look like a source of kind of mystical knowledge that made computers seem cool. And so the web kind of in that sense also arrived at a pretty specific time where people were ready to be excited about technology in that way. Now, here's the thing. Larry Wall created Perl in 1987.

That's four years before all this excitement about the World Wide Web. The Perl that Larry Wall first gave the world was just this general-purpose Unix scripting language. Totally helpful for sure. But Perl also had these secret elements that were going to make it an ideal language for a world of web development that was just around the corner. Quite famously was invented in a kind of secret NSA project that Larry Wall was working on. Essentially what he was doing was creating a hacked version of News,

the software that Usenet ran on. And so the language from the beginning was about text manipulation and moving data from one spot to another. And this just completely fit in with what was going to be needed for the web. And Perl, as the scripting language that was easy to use, that was more closely resembled natural language, it could be used to develop things quite quickly. All these things made it a perfect fit,

not just for the professional users, but also this kind of new group of amateurs that came in. Almost by accident, Pearl was ready to capitalize on the web when it came along. Larry Wall couldn't have known that the web was about to show up. But when it did, things dovetailed. But here's something else that I think is key. Perl was a free language. Larry Wall released it under the GPL, the general public license that had been developed by the Free Software Foundation. Wall's decision to make his language free was totally in

line with a new web-based way of thinking that was just starting to emerge. And Pearl was forward-thinking in other ways too. Pearl was incredibly open in a sense that it was always willing to integrate something else, something new. And this really goes with who Larry Wall is. He's a very open person who will kind of, he's very humble and, you know, will always sort of carefully consider what others think and try to put himself in their shoes. And it's interesting how, for me, Pearl's kind of character as a language and as a community

really fit that for a long time. Again, Perl was ideally suited to that early Wild West phase of the web, that hacker moment. In fact, one of Perl's mottos is, there's more than one way to do it. There's a kind of almost romantic connection between, you know, the sort of heyday of Perl and the heyday of this, you know, sort of experimental open web before it became so dominated by just a couple of platforms. Remember that the 90s was a moment in the web's history when people were still rushing to figure out what was possible. There was this huge new need for programming. Everybody had to have a web page, and that meant a whole new army of developers.

All of them open to new ways of doing things. And the question became, okay, we've got a brand new territory to play in, but what language is going to get the job done? Pearl didn't turn out to be the final answer to that question, but for lots of people, Pearl was the first answer. I'm not saying that, you know, I would rather have, you know, super slowly loading web pages and no Google search engine, but there is something, of course, really nice about that time when a couple of guys would, from their dorm room, create something like Slashdot. There's something special about that time

that I do think that as the web has become so mainstream and professionalized and so concentrated around a few big companies that we do miss. And for me, Pearl symbolizes that kind of more than any other story from that early period. Michael Stevenson is an associate professor in media studies at the University of Amsterdam. So, as the 90s rolled around, Perl emerged as a language that fit in with the early web's possibilities. It was a language of its time. Larry Wall, and Perl by extension, got what the web was all about.

You could do searches. You could create pages on the fly. And this was, you know, the brave new world of the Internet, of the web, that you could do this stuff in real time. That's Mike Bursell, chief security architect over at Red Hat. Mike was one of those hackers who discovered Perl was the starting point for many people. Java was early days. It wasn't really fantastic at text input output. And if you wanted to be taking queries and generating pages, well, Perl was what people were using anyway. Perl was just a good fit because it's good at taking text and doing other things with text.

And that's what the web was. Larry Wall has a background in linguistics, by the way, which explains Perl's strong text parsing. And that was a huge bonus, as Mike Bursell mentioned, because the web was largely a text-based medium in those early days. People didn't really have the bandwidth for graphics. So it was easy to use, it was easy to copy, people were very open with sharing, and it produced output quickly. And that was all good stuff. Oh, and the other thing, of course, is you could pipe things from it. So it's very much what people are used to and very easy to test, even offline. And that was all very useful.

Useful, especially for all those sysadmins who were reimagining their lives in a web-based world. Perl was a real gift for sysadmin. Even those days, you're getting lots of logs if you're doing anything interesting. And managing those logs and breaking them down and searching them and being able to present them in different ways or taking any other large corpus of text, which is basically what logs are, or even debug,

unless you were going to sort of pipe awk and sed and stuff and ed on the command line, which gets very painful very quickly, Perl was your go-to for that. By the late 90s, Perl 5 had pulled together a powerful community of users. Older languages like Fortran and C

had relied on big expensive hardware, but Perl was scrappier and more portable. It was designed to thrive in a world where hardware costs were dropping dramatically, letting all those new programmers work fast and easy. That's because Larry Wall made his language more human-like at the expense of CPU and memory requirements. All those elements combined to make Perl a language that welcomed the new development community. There was very much a growing community of people out there who you could go and talk to about what there was. Perlmunk started to arrive. It was a great place to discuss things, find out what was going on. That community really did have the best the web could

offer. They discovered a huge repository of software modules, a bank called CPAN, which those sysadmins loved. It gave Pearll even more flexibility. Lots of people could deploy code written by a few superstars. It had libraries to do pretty much anything you wanted. And if you couldn't find it, you asked and someone wrote it. Mike Bursell is the chief security architect at Red Hat. So, Perl was free. It was supported by a growing library of modules. It was portable, and it had a booming community. Things were looking good.

Perl could exploit all the best in that new 1990s world of web development. But just as the 90s came to a close, as if on cue, the development landscape changed again. Changed big time. But how do we know when irrational exuberance has unduly escalated asset values, which then become subject to unexpected and prolonged contractions. Irrational exuberance. That's Alan Greenspan, then chair of the Federal Reserve, talking to the American Enterprise Institute in 1996. And that phrase of his, irrational exuberance, was a warning about

the dot-com bubble everybody rode through the 90s. All those early web developers using Perl were riding high on that bubble. But it all burst in 2000, just like Greenspan predicted. In the wake of that burst bubble, the world of development realigned itself, and room opened up for newer, slicker, faster, better adapted programming languages. Perl, which just a few years ago had been the darling of developers, started to feel like a first wave tool tool in a second-wave world. New options started appearing. Hi, I'm Connor Myhrvold. I've been programming for the last five or six years

and I'm currently in the technology space working for Uber. When Connor was in high school in the early 2000s, Perl was still a pretty big deal. But he was increasingly aware of a competing language called Python. One thing that Python had going for it is it's a much more structured language and that there's one obvious way to do a lot of different things. And it's set up that way. Whereas Perl kind of relished in being the fact that there's more than one way to do something. But that was confusing to a lot of beginners. Perl had that motto, there's more than

one way to do it. Python was actually invested in the opposite. Python gave you one obvious solution for each problem. And that meant looking at somebody else's Python code was easy to do. Looking at somebody else's Perl code, on the other hand, could be confusing. Perl made sense as a person's third or fourth language precisely because it was a scripting language. It was the duct tape of the internet.

But Python was a language you could really dig into, even as a newcomer. Perl had a specific set of strengths, searching for text, bioinformatics. But Python was this easy, general-purpose language. Python gained more and more cred as the language people wanted to learn first. That was a huge deal. A lot more people were going online and a lot more people were learning how to program. And so one thing that Python benefited from, especially relative to Perl during this time period, is that it was relatively easy to learn because it was more structured.

And as a consequence of that, in a world where the pie is increasing quickly, if you can get more of that increasing pie, that will eventually mean that there are more tutorials and more stuff available to you. I mentioned CPAN before, that awesome central repository that Perl users could pull from. It was a major draw in the 90s. But the value of CPAN was also changing. That also doesn't really help you learn a language because you're kind of copypasta-ing and just replacing what you need to in the minimum amount of fashion. That became a disadvantage over the long run because if you have people that learn

how to use a programming language by doing original development of everything themselves, even if it takes a lot longer, they feel like they're more invested in it and they understand what's going on. Python didn't have a centralized repository like CPAN. But the kind of developers coming to the table in the new millennium, in a world where search on the web was so much more powerful, that repository didn't have as much value anyway. Python eventually had loads of tutorials and, of course, today you also have platforms like GitHub. And so what you ended up happening is a lot of advantages that Perl had were these kind of network effects from an era that kind of was outgrown. Connor Mierwald is an engineer at Uber. Languages rarely rise and fall by external forces alone, though.

And the internal issue over at Perl was that as it was evolving, it seemed to hit a roadblock. Python was releasing new iterations in a pretty orderly fashion. But over at Perl, well, as we learned in episode one of this season, in 2000, that dot-com bubble was bursting, and Python started gobbling up new developers. Everyone was expecting the release of Perl 6, and people were excited. They waited and waited and waited.

They waited for 14 years. People came up with about 300 plus things that Pearl 6 should be able to do. And of course, a lot of these things were basically mutually exclusive. That's Elizabeth Matt Tyson, who works as a core developer on Perl 6. Elizabeth was at the Perl conference in Monterey in 2000. That's when developers decided they were stagnating and Perl 6 was necessary. Larry Wall agreed, but if Perl 5 was his rewrite of Perl, he wanted Perl 6 to be the community's rewrite of Perl. And group work can take longer, even 14 years. It was a long and difficult road for those developers. You could argue that the current implementation of Pearl Six is actually the third attempt of implementing it.

The way Elizabeth tells it, there were attempts throughout those 14 years. Attempts that had long, agonizing deaths. People burnt out. People ran into dead ends. By the time Pearl Six finally arrived on Christmas Day 2015, much of the world had already moved on. And it's important to note that Perl 6 wasn't ramping up into some revolutionary new thing that would strike back at Python. Perl 6 was more a thoughtful reworking of the original. I think Larry Wall actually used a very nice metaphor in one of his State of the Onion speeches.

Pull 5 to him is like The Hobbit, and Pull 6 is like Lord of the Rings. If you really look at the stories of The Hobbit and Lord of the Rings, they basically are more or less the same story. It's just that The Hobbit is much smaller,

has more plot holes than Lord of the Rings, and isn't as extensive. And I think that describes very well what the difference is between Pearl 5 and Pearl 6. It is basically, it's the same idea, it's the same mindset, it's the same environment, but a re-imagination of it. Elizabeth Matthyssen is a core contributor to Perl 6. Today, Perl might not even place on a list of the top 20 languages. Between external competition and internal delays, it just hasn't proven itself for most new developers. But that raises a big question.

Is popularity really how we should judge our programming languages? Or should we be looking at other values? What does it really mean when the megastar becomes a sidekick. Around the turn of the century, as the dot-com bubble was bursting and Perl's dominance began to wane, Larry Wall made an interesting statement. He argued that, while Perl would never again be among the world's most popular programming languages, it could be a leader within a smaller category. And that, said Larry Wall, was the real goal. To be best in category, not best in the world. An SUV, he said, would never compete with a race car.

I wanted to dig into that idea. I wanted to understand what best in category really means for a programming language. I am John Syracusa. I am a programmer and I'm a podcaster. John actually co-hosts three podcasts, Accidental Tech Podcast, Reconcilable Differences, and Robot or Not. We got chatting about Pearl's position in the world today. What about today? Does it still hold up as being best in class? Perl 6 has things that no other language has

that other languages should have. And I keep waiting for other languages to steal from it. Grammars, for example, is a great way to conceptualize a common task and using grammars to solve parsing problems is, in my opinion, more pleasant, cleaner, and nicer than using the existing tools that we have. There's lots of parts of the object

system in Perl that might seem frivolous, but that I fully expect other languages to eventually adopt just as many of those languages eventually adopted a lot of the ideas that were in Perl 5. So I think Perl 6 is best in class in many categories. It's just such a shame that so few people will find themselves with an opportunity to use it. So what do you think needs to happen for the Perl 6 community to grow? What needs to happen for people to be more involved in Perl 6? It's kind of been the story of Perl 6. It's always been looking for a really fantastic implementation. That was part of the sort of second system problem, second system syndrome I think they call it, of Perl 6 where you want to fix all the problems of the world. One of the problems they wanted to fix was the runtime.

What is it that runs Perl 5 and Perl 4 before it is a gigantic C program written by people with very idiosyncratic coding styles and a ton of macros. It's kind of inscrutable. The idea with Perl 6 is let's not do that again. Let's not make a giant wad of C code. Instead, let's come up with a virtual machine, which was the fad at the time. And there were so many grand ideas about how it was going to work. In the end, we got several medium to okay implemented versions of virtual machines, sometimes competing with each other, none of them really delivering on the performance and stability and features that the

language needed to support it. What is actually going on with Perl today? What's your take on it? So Perl 5 definitely seems like it's on the downward slope because all the other languages that are contemporaries with Perl 5 took most of its best ideas and have gotten more support. That said, because it was the king for a long time, there's a lot of Perl 5 code out there running some big important sites um and people need to maintain and expand that code so it's going to take a long time for that to fade away just look at cobalt that's still out there and how people are still hiring people to work on cobalt right and that's that's you're asking if it's a dying language and i mentioned cobalt it doesn't

sound good because will pearl 6 itself ever become a dominant language, it's not looking good. There's so much investment surrounding the other languages. And if Perl 6 hasn't gotten that investment now, I'm not sure what's going to change to make it catch on. So if you had it your way, what are your hopes for Perl? What would you like to see in Perl 5 or Perl 6? And what would you like to see happen in the future? My hope for Perl 5 is that people don't dismiss it, because despite all the other more popular languages, there are still problems today to which Perl 5 is the best solution. Very often, those are sort of glue-type problems. If you find yourself ever writing a shell script, and you're like, well, I'm not going to use my quote-unquote real programming language for this,

whether it's Python or Ruby or whatever. But a shell script can let me connect together a bunch of stuff. Perl is a better tool for that job. It is easier to write a correct Perl script than it is to write a correct shell script. I think what it all comes down to is that Perl might not be the entry point anymore.

But for a more seasoned polyglot developer, you never want to throw out that niche tool in your toolbox. The specified tools are sometimes what allow you to level up. Sometimes I'm sad and depressed about Perl 6 and think it's going nowhere. And sometimes I'm like, you know what? This is a nice little community and every community doesn't need to take over the world and be the language that dominates the entire industry. And maybe you can just go along like this essentially indefinitely. Like, what's the great thing about open source

and programming languages? No one's going to make you stop. You can just keep working on Perl 6 the same way you have been. John Siracusa is a programmer and the co-host of three tech podcasts. Languages have life cycles. When new languages emerge, exquisitely adapted to new realities, an option like Perl might occupy a smaller, more niche area. But that's not a bad thing. Our languages should expand and shrink their communities as our needs change. Perl was a crucial player in the early history of web development, and it stays with us in all kinds of ways that become obvious with a little history

and a look at the big picture. Next time on Command Line Heroes, what turns a language into a standard and how might a new standard emerge in a world of cloud-based development? Command Line Heroes is an original podcast from Red Hat. If you want to dive deeper into the story of Perl or any of the programming languages we're exploring in Season 3,

head on over to redhat.com slash commandlineheroes. Our webpage is packed with bonus material for you to explore. I'm Saranya Dbarik. Until next time, keep on coding. Hi, I'm Mike Farris, Chief Strategy Officer and longtime Red Hatter. I love thinking about what happens next with generative AI. But here's the thing. Foundation models alone don't add up to an AI strategy. And why is that? Well, first, models aren't one-size-fits-all.

You have to fine-tune or augment these models with your own data, and then you have to serve them for your own use case. Second, one-and-done isn't how AI works. You've got to make it easier for data scientists, app developers, and ops teams to iterate together. And third, AI workloads demand

the ability to dynamically scale access to compute resources. You need a consistent platform, whether you build and serve these models on-premise, or in the cloud,

or at the edge. This is complex stuff, and Red Hat OpenShift AI is here to help. Head to redhat.com to see how.