CoRecursive: Coding Stories - CPAN - This Day In History

Episode Date: August 1, 2022

CPAN was the first open-source software module repository. And on this day, Aug 1st, in 1995, CPAN was first announced to a private group of PERL users. If you are building things today by pulling in various packages from various open source places – and really, who isn’t – then the history of how this world came to be is essential. Episode Page Support The Show Subscribe To The Podcast Join The Newsletter

Hi, this is Co-Recursive and I'm Adam Gordon-Bell. Each episode is the story of a piece of software being built. And today is all about something I've never used before, but that has shaped so much of how software development is done today, it's almost hard to imagine a world without it. Can you guess what it is? It was first announced to a private group on August 1st, 1995, 27 years ago. And eventually the ideas embodied in it spread to nearly all programming

language communities. I'm talking about what led to our modern world of open source software development and to the world of building things by gluing packages together. I'm talking about CPAN, the Comprehensive Perl Archive Network. Maybe you've heard of it, maybe you haven't. But CPAN was the first open source software module repository. CPAN inspired everything that would follow. NPM, Maven, Cargo, Nougat, Hackage, RubyGems, Python, PyPy, and so on and so forth. There's so many.

If you're building things today by pulling in various packages from various open source places, and really who isn't, then the history of how this world came to be is important. Let me say it again. CPAN should be something you know about. It was the original. It changed software development and it has lessons to teach us about community and about collaboration. So today, I want to share the early history of package management and how it shaped the world of software development. And to discuss that, I have my former colleague and all-around car guy, Don McKay. Hello. You know, I'm not that all-around car guy, Don McKay. Hello. You know, I'm not that much of a car guy.

Listen, if you've aftermarket modified your car, you're a car guy to me. Maybe like half car guy. And also, I have my peer review is BS friend, Crystal Mon. Yay. Thank you so much for inviting me. And I'm really happy to be here today. The cool thing about Perl is that it's kind of like the story about the fun side of computing, like people playing around with things. If you go back to 1977, Unix existed and it had the shell,

which meant that you could just get things done interactively by typing commands and piping them in. You didn't need to write big programs in C. You could do like interactive stuff, which was a big deal at the time. And Brian Kernahan apparently used to do demos of doing spellcheck at the command line back in the 70s. Somebody would take a document, a text file, and he would at the command line like chain together, splitting it up into words, and then joining those words against dictionary and returning all the words that weren't in the dictionary. Bam, I made a spell check, you know, just like typed it out in a couple of commands, right? And so that was scripting, right? Like people now talk about like, oh, scripting languages

versus like non scripting languages. And it seems pretty murky. But back then it was like a big difference. So Brian and his colleagues, this Alfred Aho and Peter Weinberg, they wanted to extend what you could do in the scripting mode and make computers more expressive. So they created this language, awk, where it was easy to parse text

and where you didn't have to declare variables ahead of time. And you use like dictionaries, you know, like key values instead of arrays. There was programming that looked a lot like modern Python or Ruby instead of the C programming that was more common in Unix at that time. So that's 1977. But then you fast forward, right?

Not many people had computers in 77. But you go to 87, 10 years forward, when I was actually alive, although quite small. And Larry Wall announced Perl. And so he wanted to follow in this AUK tradition. He wanted to make something so that it was easy to kind of do these little interactive computer tasks that people had to do. I did a mock interview once in AUK. It's like AUK people use it quite a bit. So Larry Wall, he wanted to follow in that tradition. And so he announced Perl and he described it this way. Perl is an interpreted language optimized for scanning

arbitrary text files, extracting information from those text files and printing reports based on that information. It's also a good language for many system management tasks. So it was 1987. And the one thing that was different from 1977 was that the early internet culture was starting to become a thing on ARPANET. So Perl became not just a language that was a replacement for mock, although it did become that, it also became like a community. Computers were becoming more and more common. And so more people had jobs where they needed to like maintain computers or do things with them, right?

And also one year later in 88, IRC was created, which became like also an important part of the Perl community. I spent a lot of time in IRC. Were you a nice person on IRC? Nobody's a nice person on IRC. We used it for a purpose, right? Like you had a certain group that you were hanging out with and it was just a way to connect with people that were not in your local area, but yet shared your same interests.

Because like growing up in a, like a smaller town, like not like a village or anything, but not like a big town. There weren't a lot of people that were interested in video games at the time that I was going to like high school. It was mainly all sports. So you had like a few friends around,

but there was no bigger community until like IRC. And then there were channels for that kind of thing for people that were fans of certain things. And you could connect with those people and talk with them. And yeah, I mean, anytime you introduced anonymity to the situation, you're going to get a lot of people that, you know, commit questionable behavior. What IRC channels were you on? So it was, yeah, it was mainly around Quake. And then there were, there was one for D&D. I used to play D&D over IRC. So IRC, much like it became an important part of Dawn's Quake community, it became an important

part of the Perl community. But this was 1988 instead. And also newsgroups were a thing. It could maybe troll each other on IRC or share code, answer questions and stuff on the newsgroup. Did you guys ever use news groups? I don't recall using news groups at all. Like very early when we got the internet, I think the email program that we used, you could sign up for news groups. And then I did.

And then I just remember that it would take forever when you went to check your mail because it would download all these threaded conversations of of all these people like talking in depth. And like, I didn't know what they were talking about. And so I think I bailed out of it after probably using up a lot of our bandwidth, just like downloading old, like long threaded conversations. This was before the World Wide Web existed, like ARPANET existed, but there wasn't websites, right? Mainly people at universities or research institutions, you know, like using the internet for email and newsgroups. And I guess like FTP existed, had for a long time.

So people would talk and argue on IRC. But then the web came along and websites came along. And so people started sharing, you know, these bits of Perl code that they were exchanging, like on their websites or on the news groups. It was sort of a new thing that people could meet with others like this and talk about a language remotely. And this isn't like commercial software.

This is just people trying to get things done and sharing stuff. And this went on for a while with more people hanging out online, more computers coming online, people talking and sharing code and then in 1993 is when mark andreessen created the mosaic web browser and then the web started to really be a thing and then there was the birth of cgi scripts you guys know what cgi scripts are it was like the beginning of web applications yeah because like before then websites were like you returned this static document yeah like an like an html page right like you just wrote the html and that's what you but like the cgi scripts would be like the way of using like an application

like code to like generate a response it was revolutionary at the time right it was like a different way of thinking about things it was proposed on a news group as well. And originally it was like writing programs in C, but then quickly people started using Perl. They started making these Perl scripts, these CGI scripts. You could hook it up to the web browser, up to your web server.

And now you had dynamic content. And this was really cool. Like web pages before then had been static, but now someone could share like a Perl script, like returned an image of a counter, right? And then this became like the web counter. Like when you went to a web page, it would fire off this Perl script that would return an image and say like,

oh, you're user 35 to Adam's awesome website. This many hits, right? It was all about hits. And they have the map where it's like, it's from all over the world. Yeah. Mostly from the US. And so someone made a Perl script that was like a contact us form. So you could fill out a form on the web page. Right.

And then it could send off an email to the owner. Or I don't know if you remember guest books. Do you remember that was the thing? Yes. Oh, my God. I was not always a very good guest, though. What did you say? If their website was kind of crappy, like you kind of want to be like, this sucks.

That's awesome. You were trolling them. You're like, this sucks. Yeah. Guessbooks were a bit before comments and articles. It wasn't like, oh, I read this and I liked it. It was more like about your whole website. I was browsing the World Wide Web and I came across your website.

And you mentioned that you went to St. Mary's School. Did you happen to go to school with my cousin, Roger? Like say hi to Roger. Like it was just like a weird, like a weird world, right? Yep. I've always felt like it's suited people who collect weird things. Like somebody's into like birds or like weird stickers or stomps or something. And they're like, this is my website where I like just show all the things I have and talk about infinitely about things that most people don't care about. And some other person would be like,

I'm a weirdo about those things too. And they would leave, they would sign the person's guest book. You do needlepoint of old beer labels. I do that too. Like we're the only two people in the world. Did you have an early website, Crystal? I think I had a MySpace. I don't even know. How about Don? Did you have an early website?

Yeah, I did. I used Macromedia Dreamweaver. Oh no. Nice. I had it for my D&D game. So I had a D&D group and I used to like post it up, not for like people on the internet, but from like our friends group. You can go on to the website and you can, if you miss a game, I would take all the notes that I took from the game and then type it up as a story of what happened in the last game. And then I would post it up and so that everyone could read it so that it became just one long story. And I had a website for that. That's awesome. So what happens, right? People are sharing these scripts, like here's how you make a guest book or here's how you make whatever. People start sharing those and then other people start collecting them, right? This is like valuable information. You know, I remember when I first got Napster and high speed

internet and I was just like, I need to get all of the music. Like that's just what I need to do. And I think it's like a common thing, right? Where you're like, this thing has value, let me collect it. So there was this mailing list that got set up for Perl people who were collecting code and they called themselves the pack rats. So there's a mailing list. I don't know their original motives, but they were collecting, you know, various bits of Perl code and they had FTP sites and they could share it around. They said, you know, we should pool our resources.

We could mirror each other's Perl scripts on each other's FTP sites. Okay, so how do we do this? How do we divide the various scripts? How do we organize it? Should it even be FTP, right? The web is a thing now. Maybe we should use that. There was Gopher, which I think I never used, but I think was from before that time. So they decided to call this CPAN after CTAN, which was the tech archive network for tech and LaTeX. Crystal, what is tech? So if you're trying to write like the sum over N starting from one, it's hard to write as in a document.

Like it renders the mathematical notation really easily. I actually use it like every day and I love it. So in this private email group, there's like an email where somebody said like, let's do it, let's do this CPAN thing. Let's find a way to organize all these piles of code that we have, right? And around this time, also Perl 5 came out

and it had this feature called modules. So you could organize code into packages. It didn't just have to be like, here's a guest book. You could say, make a package that was like, okay, this does whatever. It figures out how old people are if you give it your birthday. So they put together this master list. So the master list was a list of modules and where you could find them online. So like Don's birthday calculator, and then here's the link to Don's FTP site.

And then you have like crystals, like token ring module that's on crystals, FTP site, and so on, right? You could imagine building a script that goes through all these grabs, all these things and puts them on your FTP site. And then you're running a CPAN mirror. So they came up with this idea that they would create a place for people to put their modules. So this they eventually called PAWS, the Perl Authors Upload Server.

But there was a guy named Andreas J. Koenig, and Jarko Heinemainen. I don't know. He has an H name. So they were the big people. There was like several people on the list, but these two people kind of did a lot of the development and a lot of the early effort.

And a guy named Tim Bunce, he made this original module list. But so building all this took time. They were going to build POS, which is Perl author upload server. And you could register and reserve a name, right? And then once you had a name, you could just upload your module code there. So Don could upload his token ring stuff and Crystal her birthday thing. Is that how I said it? Yeah, I know. That's the other way around.

And then, you know, CPAN could mirror all this. Like it would all be in one spot. And then one of could mirror all this. It would all be in one spot. And then one of them got this idea, why don't we just, instead of just listing it by name, that's not really the way people use software. They're not like, hey, what's great new software by Dawn? I should check out, right? If you're trying to find a module to do something,

you should categorize it. And so they came up with this idea, like, let's do categories. So they built a tree of categories that went from just being like, you know, sort of like an old FTP thing with people's names and files in it to them saying, like, oh, here's a section for all the code

that talks to databases. Here's a section for all the code that talks to user interfaces. And here's file handling. And here's the World Wide Web and all these collections of software. And it was still stored under people's usernames but it's just kind of they like mirrored it under the different category names so it's easier for people to find but yeah it's it's interesting how simple it is like if you just list by username it's sort of like github

right you go on github you can look under people's usernames. But once they start organizing it by like modules, this is something cool. I feel like you could go full bore with the whole pack rat thing and have like, you know, they had like a logo with like a one-eyed pirate rat, you know, and who was like in a box and, you know, like they had like t-shirts and like a whole thing going. And I don't know.

I feel like that's what drives a community is like online communities as well. It's like that's what drives a community is like online communities as well. It's like, let's do a thing together or like. No, I totally agree. So yeah, on August 1st, 1994, they announced this. They're like, we built it. They told some other Pearl people. And then on August 16th, the first package was uploaded. It kind of took off. You know, they let it have some time to bake. And then on October 26 of that same year, 1994, they announced it publicly to all the Perl people on the Perl news group. And then things changed because all these people could upload modules.

And yeah, like you said, Crystal, like I love this idea. There's like, you know, there's these bunch of sysadmins or whatever they do. They're working at various places. It's still early in computer history. So they're probably in charge of the computers and between keeping everything running, right. And their nights and weekends, they were building this thing. They were building like, here's a way that we can collect all of this Perl code. And it wasn't even until 1997. Like, what is that? Five? No, my math is not so good. Three years later. It wasn't until 1997, like three years later, O'Reilly had a Perl conference. They brought together all the Perl users. And that's when,

like, especially these two people, they first met each other. Like they were just strangers building this community, this little mailing list. And then they got to meet each other in person years later. So this little group and this little mailing list, but really these two people, they built this thing that really changed things. So here's user Melling. CPAN was Perl's claim to fame. It's what gave it an edge over Python and Ruby.

And then here is user Cerniman. Many thanks to the Perl community for building CPAN and having an easy-to-access repository of reusable libraries. It's given us more power to develop cool stuff than any specific language features. That's why nowadays every important programming language has some sort of CPAN clone. And so CPAN and Perl, they were a community, and people kept contributing to CPAN. And so by 99, there were around 200 packages a month being released on CPAN. And by 2001, this crossed to 500 releases per month and it kept on building. Sometime in 2004, there was a thousand monthly package releases.

And some of these might've just been like minor upgrades of existing packages, but like a thousand monthly package releases. And some of these might've just been like minor upgrades of existing packages, but like a thousand a month is a lot, especially for 2004. And one thing that happens when you're early like this, like you build this community and it's early, is you hit problems that nobody else has had.

Nobody's hit them before. And one of the first problems they hit was like, how do you know if one of these modules is good? If it even works, right? If you have a thousand of them, thousand releases a month. So they created something to deal with this. Perl users had, you know, they had various versions of Perl. They all had various OSs and various computers, right? So they had something in the Perl modules. You could run make tests to run the tests. People would just download modules and test them and report like, hey, I'm on this operating system,

I'm on this pro version, and it worked or it failed, right? But obviously that doesn't scale very far if you have like a thousand releases a month. So they had to come up with an automated option. So people would volunteer machines and then on that machine, it would download modules and run tests and send back to this website called CPAN testers, like how things worked. So it was in fact a global CI, continuous

integration server for packages and CPAN, but it was distributed, right? Because people could volunteer their computers to just download all these packages and test on all these versions. Then you could go on CPAN testers and see like, oh, this works on this and this and this. It must be good. It's kind of wild that they built that back then. CI is standard now, but I don't think any package repositories out there

are like testing every release across all these various versions. Now these are commercial entities, right? These are just a bunch of volunteers, just a distributed system of volunteer computers, you know, testing all this code. And then they added other things on top to deal with the scale. They added ratings for packages. They added a website that would index them. And so you could search. So if you're searching for whatever you needed to connect to ICQ via Perl, you know, probably somebody has a module like that and you could search and find it. Were you on like ICQ and stuff back then?

Uh-oh. Uh-oh. What? Oh, you don't know those ICQ noise? Oh, the noise. Yeah. Oh, sorry. Yeah. Yeah. So it's such a cool, small community working towards a common goal. They're all coming together and building this stuff. And it really became Perl's advantage. As other languages appeared, they wouldn't have CPAN. So if you needed to do something, there was already a way to do it in Perl. And it was probably tested and it probably worked. If you want a random Chuck Norris saying, if you want to read some odd file format, right, probably CPAN already had something that handled this. And we kind of take this for granted right now, right? I feel like every other language community, you can kind of expect this. But everybody got there by copying CPAN. Like this was something magical that they came up with. And I'm sure at the time they were like, I think we're onto something here. You know, like they must've felt it.

Like, look at what we've accreted, all these packages that do all this stuff. So in my world, right? In my personal timeline, after my GeoCities website and after I went to university and we played with lots of languages, but really a lot of it was a Java.

And I remember building stuff in enterprise Java, like enterprise Java beans, EJBs. And it but really a lot of it was a Java. And I remember building stuff in enterprise Java, like enterprise Java beans, EJBs. And it was like a lot of XML and it was very kind of crufty, but I thought it was so cool. Like enterprise Java beans was like an open source thing. You had to like download like Apache Tomcat. And for my final project, I built this EJB point of sale thing for like a video store,

like a video rental store. Like apparently the video rental store concept is more dated than the EJBs at this point, I think. EJBs were open source. It wasn't really like CPAN. It was more complicated. You had to get Apache Tomcat and like get it running. So I leave university and I get a job.

And at the job, we're doing C Sharp development. And C Sharp is really new at the time, right? And in Microsoft, just like in Java, there was all these cool open source efforts. And a lot of them were just porting Java code over to.NET because.NET was new and Java existed. So there was Hibernate was like a big persistence framework in Java. And then they brought it to.NET. It was called N Hibernate, was like a big persistence framework in Java. And then they brought it to.NET. It was called N Hibernate. And there was JUnit, was like a testing framework. And then there was NUnit. And there was Ant, which was Java building.

And then so there was NAnt. Don, did you use C Sharp back in these, it's like C Sharp 1 or something. I don't know if you were involved back in those days. Yeah, no, I learned C Sharp in college. there was a class, it was a sixth semester class about C sharp. Yeah. So like I got this job, we were doing C sharp stuff and I was learning C sharp and I loved it. Like it had this great, like visual studio was like so much better than all the Java things I would use. And there was all these bloggers and they would be like, yeah, like I created like NUnit and isn't it awesome?

Or I'd learn how to use Nant to like build things. But when I mastered it and I would learn like, oh, here's how Nant works. And I would, you know, there's all the community people explaining and it felt cool that I was involved in it. Then Microsoft was like, oh, we came up with this thing called MSBuild.

It's like exactly like Nant, but it's better or something, right? And then everybody would switch. They would abandon this like open source and like the mailing list would die. Like who's going to use Nant now? We've all moved to MSBuild. I didn't get to use Nant, but I am very familiar with MSBuild. Yeah, I've heard of MSBuild as well.

That's so frustrating. So then there was NUnit, which was the unit testing framework. And then there was this one that was called MBUnit. And the guy who built it was like brilliant. And it could do all these different kind of testing things.

And I thought MBUnit was so cool. Then this guy, he got a job working for Microsoft. And then Microsoft came up with this thing called MSTest, right? And it was a lot like NUnit, but didn't have all the cool features got a job working for Microsoft and then Microsoft came up with this thing called MS test. Right. And it was a lot like N unit, but didn't have all the cool features of MB unit. It was just like a standard way to run tests. And the cool thing about all the open source ones, like to me was like, there was these people with blogs and they were like describing how it worked or there'd be

like, you know, like mailing lists and, and people chatting about it and this like community, right? And slowly in my time in early.NET, it just seemed like an act of war. Microsoft was crushing all these open source projects. There'd be like a very cool, very talented open source programmer building something and like community would build up against them. And then like Microsoft would be like,

oh, we've built a version of that. And then everybody would just switch to that because, well, it was Microsoft. You were already bought into Microsoft if you were using C Sharp and.NET, right? So if Microsoft comes out with a product, you're like, well, I'm not going to use that open source one by just some guy on his blog.

I'm going to switch to that. And I struggled with this. Like I got mad about this. I started working with UDOM, but I was building some of my own stuff on the blog. I'm going to switch to that. And I struggled with this. Like I got mad about this. I started working with you, Don, but I was building some of my own stuff on the side. And a lot of these like open source.NET people, they formed this community that they called alt.net. Like it was supposed to be like alternative. They were like feeling what I felt, but probably even stronger because they built these tools. You know, we want to be.NET developers, but we want to use the things we built. We want to use our open source tools

and we want to support them. And we're like an alternative world, right? It was like this CPAN world that CPAN started was trying to blossom in the.NET community. Intentionally or unintentionally, it was being crushed. It was being squashed. You can't build up a community around Microsoft releases MS tests and put some documentation online. Like, that's not a community. And then in 2014, they make.NET Core open source. Yeah, I mean, only a decade later or something. And you couldn't even see the source of the code you were using in the.NET framework.

You couldn't go into the source code for the methods you were calling and see how it worked. You couldn't learn from it. You could just consume it. It was a black box. You send things in, you get things out, and that's it. Yeah. And like Microsoft has changed, right? As Don said, they've now like embraced this model.

I mean, I'm not part of the.NET world anymore, but they've changed very much. And the reason I bring this up is because I feel like there's an alternate world. There's a different timeline where, you know, CPAN didn't happen. And this model that Microsoft had of like handing everything down, we could live in this world where Microsoft or Google, they build all the software packages

and everybody who builds software is just like a consumer of these black boxes and how software works is handed down to you and you don't get to understand how it works. And I'm glad we don't live in that world. Yeah, there was a big push for proprietary solutions early on. There was a lot of pushback from within Microsoft as well by prominent developers.

Even though they work for these companies, they're part of those communities too. You know, like they're bringing back the information to Microsoft about, hey, there's this cool package or this cool library that this person just made. And I don't know. I find that to be incredibly frustrating.

Oh, we'll just wait. It's going to get worse. I'm not against people making money, right? But like they were against people freely sharing software. The thing I didn't like is it separated people into hierarchies. There's the people who are building the important libraries who work at Microsoft, if you're in the.NET world, and then there's the consumers of it.

You're either part of that or you're not. This is the awesome part. Microsoft used to have these personas and they publicly published this, which seems like a horrible idea. So they had built marketing personas of the developers who use their software. They had three separate personas. All of them were dudes. That's actually not the bad part. So they're all men, but they were called Einstein, Elvis, and Mort, right? So Einstein was a genius to their description, right? And he built everything himself and he cared a lot about performance. Elvis, whose picture was literally of Elvis Presley, was a pragmatic developer, right? And then Mort, Mort was dumb, right?

You could just tell, like, I feel bad for people named Mortimer, but they're like, Mort just does whatever he needs to do to get his job done and then goes home. It's very demeaning to all these people who are using the.NET software. Kind of what they were saying is like, oh, there's like the C++ people who really get into the performance and stuff and understand how computers work. And then there's Mort, right? It's like VB.NET, C Sharp, here's Mort. Fuck that. That is so cringe. Yeah.

We're not that dumb. I mean, I feel like it also paints people as like this static kind of persona too. It's like people can start off not being super knowledgeable about a thing and eventually become experts in things. So I don't. Yeah. Just painting somebody as like a Mort is. Yeah, that's terrible. So they did receive a backlash from this.

Here is Scott Bellaware talking about it. The Microsoft developer personas that include Mort, Elvis, and Einstein are ultimately an ethically bankrupt mechanism to pigeonhole software developers into overly simplified categories that typical marketing staffer is comfortable with. It appears to be a bid by developers

to rid themselves of the capacity for rational thought in favor of tribal identification with corporate brands and rock stars. You know, it's developers building these products and they're like, hey, make sure you build this for Mort. Don't give any of those fancy options that might confuse. I mean, this is the alternative world that CPAN helped us avoid where, you know, you could be Elvis or Mort or if you're smart enough, you might be an Einstein. And then you might work for one of these big companies. You might work for Google or you might work for Microsoft, or you might just bounce out of the field because the field doesn't reward deep thinking, right? If everything is targeted

at people, you know, with this assumption that they're not smart, then what does that say, right? People will just leave. The CPAN model is different. Everyone can come. Everyone's part of the community. Everyone can contribute. I really loved C Sharp, and I really liked, especially they had the link style,

concise syntax that you could write. But the.NET community back then, it was stifling. It seemed like they didn't want makers and people who wanted to create stuff. They wanted consumers and fanboys of whatever the latest thing

they were putting out. And they put out a lot of cool stuff. But yeah, it was just... To me, it felt like a problematic scene. And I don't want to blame Microsoft for all this. Microsoft is a big place, like Don was saying. There I don't want to blame Microsoft for all this. You know, Microsoft is a big place,

like Dom was saying. There's lots of people there. And I'm sure that there was people in the community, there's people at Microsoft who wanted things open source. I mean, maybe. I mean, I feel like I'm a little bit bitter

about this. I don't know if I'm too bitter. Is it too bitter, guys? No, I don't think... There's always going to be a struggle within the company because there's going to be people who don't know if I'm too bitter. Is it too bitter, guys? No, I don't think... There's always going to be a struggle within the company because there's going to be people who don't develop, who don't really appreciate open source communities, and they're just looking for the way to increase the bottom line

because that's what a corporation does. And those people are at a constant kind of tug-of-war, right? It was October 23rd in and 2021 there was an article on the verge where there was backlash against microsoft because they removed a key feature from the dot net 6 release that allowed hot reload so you can modify your source code for your app while it's running and they removed it and locked it to visual studio 2022 which is their mostly paid product. And everyone was upset because you couldn't get it for free anymore, right?

But it's like a crucial feature that, you know, if you're a programmer, you like to be able to modify your code while it's running. It's just very convenient. And they reversed that decision, right? But yeah, like Adam was saying, Microsoft is a very large company

full of a lot of people and there's a lot of differing opinions on what that company should do. And you see that every once in a while. And I'm sure from Microsoft's perspective, they're like, hey, if somebody builds an important library that does something, they're like, oh, there's market demand for that. But from the outside, there's no community, right?

It's just killing it. There's no room for voices. If you're within Microsoft, you're still part of the community, but what It's just killing it. There's no room for voices. If you're within Microsoft, you're still part of the community, but what is your responsibility now? Like, do you think that you're still, let's say you were making an open source package and you get hired by Microsoft and they're like, we want to like close source and make money from this package. Like how much do you as an individual owe to uphold and kind of speak for the community that you came from? It's super tricky, right? And I think it has to do with how much you feel like you're part of that community. You know, like, I think the reason CPAN probably did so well in those early days is

because they, well, there was no commercial interest. It was just this Larry guy built Pearl. But also, like, the community all felt pretty tight and they weren't selling commercial software. They were all just people trying to solve problems. And like, I'm not against capitalism. I like to get paid, but it's hard to mix with this concept of these people kind of contributing and bringing things together. If I'm a dev in one community and I can produce PDFs by running like a single command and pulling in someone else's code. And then in another community, I have to go talk to somebody and we have to buy a license to a software package for PDFs

and add in some DLLs and do the licensing and stuff. Not that the people who built the PDFs don't need to get paid, but the guy using CPAN who can just pull it down, he can get more done. It might not be as good as like the paid one,pan who can just pull it down like he can get more done might not be as good as like is like the paid one but they can move faster but i think the cool thing is what happens next right if they find a problem with that pdf generating thing they reach out to whoever

created it and they suggest a fix or they submit a patch or they just file a bug right it might not be easy for them to fix it. It might be a huge challenge. But all of a sudden, he or she isn't just left as a mort. Like they're not this like Microsoft, they just consume packages. That person is now learning how to contribute. She can contribute to the community. She can join. This was the big mistake Microsoft made, right? Or any commercial company, they don't see that the community can grow a developer from one level to another, right? Like people aren't born great developers. They're created. They need the community to nurture them. And I think this is what these communities like CPAN or wherever, you know, the community springs up around these packages and then people try to build it together. And that's how you learn, right? You can look at the code, you can suggest things. And that brings you to that next level where you can build packages like this. So like the more that's built on this community, the easier it is to accomplish things because you're sort of training more and more people how to build the

type of software that needs to get built. There's just so much software out there that needs to get built. Like maybe at some point that'll stop being the case, but nobody's in danger right now of not having anything to do if they don't monetize like every single thing, right? So CPAN was the first with this idea, but it spread from there. So Python has PyPy and Ruby, which was supposed to be a direct improvement upon Perl. It has Ruby gems. And all these communities, they would have people in them that came from the Perl community

who would make noise, you know, if things were less easy to use, right? They would say like, this isn't as good as CPAN. We need to improve that. It's like a cultural idea that spread. It's like COVID maybe idea that's spread. It's like COVID, maybe, to use a bad example, right? Like once COVID infected enough people in a community, and then they would spread to other communities, and this thing would keep spreading. So the idea that we can all work together as a virus. Maybe not the best analogy. It's like a mind virus.

Anyways, here is Guido in 2009, the Python creator on a mailing list. People want CPAN. I just found this comment left on my blog. People want CPAN. People have told me this in person too, so I believe it's a real pain. But I don't know how to improve the world. Do we need more than PyPy? And so PyPy is just the package ecosystem for Python, right?

People from the Perl community would join. And Python, like in this mail thread, they broke out into a huge discussion, like you need to improve this, you need to improve that. The table stakes have been raised, right? Like everybody needed to be this good, right? So that was 2009. Another thing that happened in 2009 was Microsoft

changed. So in 2009, something appeared called the Outer Curve Foundation. And if you looked into it, it was just like a nonprofit that was opened by Microsoft and Microsoft employees worked for it. All the open source people were like, thought this was some big conspiracy to undermine open source, but it actually wasn't that, right? They were actually embracing the world of open source people thought this was some big conspiracy to undermine open source, but it actually wasn't that. They were actually embracing the world of open source. Probably some people within Microsoft got this idea going, but the world had changed. Microsoft was funding an open source nonprofit, and then they built Nuget, which was.NET's answer to CPAN. They built a package manager open source where people could upload which was.NET's answer to CPAN. They built a package manager, open source, where people could upload packages for.NET.

And Microsoft put all their packages in that too. So the world had changed. CPAN, this idea, using my infectious disease metaphor, right? It had spread out into the world. And now it had even gotten into Microsoft. That's interesting. I understand now like why there was such a big uproar when that whole GitHub acquisition thing, you know,

like there's so many developers I knew who immediately were like, I'm moving to GitLab. But having heard all of this from you now, that's interesting. And the thing is, I was a fanboy. Like C Sharp and Visual Studio and.NET were really nice. Microsoft was good and continues to always be good at building developer tools. Visual Studio Code, super good. But yeah, it took them a while to figure out this open source thing. That whole Mort thing too, like the whole Mort idea. I don't think that's necessarily something that Microsoft came up with.

Like, I don't think anybody believes that all developers are equal. There's like the opposite that I see perpetuated by a lot of actual developers, which is like, you have to live, breathe, eat, sleep code 24-7 or you're no good. Yeah. And like, that's not true either, right? There's like two extremes it's like no if you're not like crunching code when you go home pumping out some kind of cool package on your own time then you're not a good developer there's like people who can be really interested in coding and can like apply themselves at work but when they clock out they can do other things yeah like that's possible people have Burnout is like a real thing.

Like if you're a central maintainer of a package, like you can burn out too. Like just having some kind of policy too for what if someone just doesn't want to deal with maintaining a thing anymore? Yeah. How many tech things like or programming languages have I been excited about?

And then like a year later, I'm doing something completely different, right? right like thank god I didn't build something really important in one of those and I'd be like oh my god I don't remember how Idris works like I gotta maintain this thing you have to go back to Scala Adam what would you do you can't you don't remember I think I remember some things print hello world so also in, NPM was created by Isaac Schulster. He created it after saying that, you know, he had seen module packing done horribly and he wanted to do it well. And he wanted to take inspiration from projects like CPAN. When it comes to talking about packages and people sharing them, like NPM is where people start rolling their eyes and saying

like, well, that community is all messed up. But I think it has a bad reputation. So in March 11th, 2016, there was this guy named Azure and he got an email from this other person named Bob Stranton. And Bob said like, hey, you have this package named Kik. It's like K-I-K on NPM. And I work for a company called Kik, and we want to use that name. So you should give us that. And Azure said, no, that's my open source project. No, thanks.

And then Bob said the following. We don't mean to be a dick about it, but it's a registered trademark in most countries around the world. And if you actually release an open source project called Kik, our trademark lawyers are going to be banging on your door, taking down your accounts and stuff like that. We'd have no choice but to do that because you have to enforce your trademarks.

So this is the best part because what Azure says back is, you're actually being a dick, so fuck you. Yes, that's the first thing I thought was like, why say we don't mean to be a dick about it? Correct me if I'm wrong but if he had the name first even if they

had a trademark like he could prove that his name was there first. Yep. Yeah. I don't even know how it works but

here's where it goes sideways. So Bob goes to NPM who are a commercial entity. It's not quite like CPAN. They're a company.

They charge some corporations money and they say the same thing to NPM, who are a commercial entity. It's not quite like CPAN. They're a company. They charge some corporations money and they say the same thing to NPM. I assume we're going to sue you. So NPM takes it. They take it away from Azure and they give it to this Bob guy and kick. And Azure says, listen, you have hundreds of packages on NPM

that overlap with trademarks. And he was right. I mean, WeWork trademarked the word we, Square, you know, the payment company. They, you know, they try to own Square. Well, trademarks are also, they're confined within the framing of you have to mistake that for something else

that's in that particular industry. So if they trademark Square, you could have like a shoe company called Square. No one's going to mistake your shoe company for the payment processor, right? Even though they're named the same thing. But if you have something that's technology adjacent that's called Square, then Square would be like, I don't know, people will think that we're affiliated with you or that we make that product as well. I mean, NPM probably got scared of the talk of lawyers. I don't know. But anyways, what happens

is Azure gets mad because they sided with Kik. And so he deletes all his packages on NPM. It turns out he has 273 of them. He's actually a pretty big contributor to NPM. And not the most popular of his packages, but turns out one of the most important ones is left pad right so left pad was an npm library that just can like pad out a number or a string and when he removes that it basically breaks the internet because guess what uses left pad it was react right and everybody used react everybody still uses react yeah i remember. And so thousands of popular packages stopped working. I don't know how many websites, like they just couldn't build it

because they'd get an error because it would try to pull down this library that didn't exist and they could no longer build their site. That's kind of a scorched earth policy, right? You're hurting a whole bunch of people that aren't even involved at all, right? I agree.

He was probably pissed. And probably NPM should have cached these dependencies so that like everybody's website didn't break the moment he pulled it but yeah so what happens is npm they freak out and so they go in and they un unpublish his code they republish his left pad thing to like fix everything right they put it back up i mean it's only 11 lines of javascript people don't talk about this i think they broke the expectations of what this package manager was, right? Like they sided with like an outsider corporate entity rather than like the users of the community. And then the story all became about how

NPM is a mess and jokes about how like JavaScript developers have too many packages and whatever. But I think the main thing that was missed was that community part, right? NPM is big, much bigger than CPAN. And it's hard when you get to a certain size to maintain this sort of like community spirit. But it's especially hard when you feel like these decisions are going against you. You know, like you're building things for this faceless group, and then you're getting demands that somebody's going to sue you because of some like little package you made.

Imagine you just throw up some software somewhere and some lawyer's contacting you and saying, like, we're going to sue you because of this. It puts up a barrier to entry, right? Like, I don't think I really want to be part of that. Yeah, exactly, right? I mean, this problem isn't even limited to development. I mean, content creators face this problem

all the time with YouTube, right? Like, they're the ones creating the content, right? But they're also the ones living in fear from, like like DCMA takedowns and stuff like that. Yeah. And like Kik is like a second rate WhatsApp clone that's being investigated by the SEC because they issued their own like crypto coin because they ran out of money and whatever. And like there's allegations of child predation. Like they were never going to be the heroes of the story, right?

But this is just a problem of communities. It's hard to scale communities to a certain size. And when it becomes like a faceless NPM organization versus some person building something, that's just what happens. And so since then, all kinds of other problems have happened. A lot of them in the NPM world, just because it's the biggest, I think.

Simple ones are just messages asking for sponsorship. People saying like, hey, you're using my code. Guess what? I don't have a job. Please send me money. It's just a little bit annoying, I suppose. And then there's open source supply chain attacks. You could pay a popular package owner

to add a backdoor into their code or people just upload typos of popular packages and hope somebody makes a spelling mistake and then pull in a bitcoin miner or there's like log for shell log for j the java logging thing like it's just complicated enough that people found an exploit and could take over people's servers the the very latest one is protest where so to protest russia's invasion of the ukraine this person who ran the node ipc package he briefly put in code that geo located your ip and if you were inside russia it would delete all the files on your hard drive which was a really bad idea some u.s charity in russia like had their files wiped out by this. And then he quickly backtracked and changed it so if you were in Russia,

it wouldn't run and would just produce a heart. That wasn't going to fix the war against Russia aggressing on the Ukraine. It definitely ruined that charity, though. Yeah, right? We just live in a divisive world. I think that you're going to keep hearing stories like this. The world of package managers, it shouldn't just turn into this seedy world of trademark fights and paid sponsorships and people begging for money. So the question is, how do we fix that? It started as a small group of people in the Pearl world, but now these packages have become big things. How do we prevent it from going bad? That's a deep, deep question, Adam.

All of them will eventually be bought by some kind of corporation, right? No! Like NuGet or NPM. NPM is actually owned by Microsoft. Is it? Well, they own GitHub that closed the deal to buy NPM in 2020. If I put on my Microsoft's evil hat, Microsoft is slowly buying up all of the developer stuff

so they can slowly return us to the world of proprietary only software. I don't think that's what's happening. Well, no, I don't think that's what's happening. But like a lot of these software solutions are being like eyeballed by corporate interest. Oh, yeah. And they usually come up with a dump truck full of money. I mean, who's going to say no to that?

I think that there's still a lot of open source repos out there that aren't controlled by corporate interest, but they're a dying breed, right? Yeah. So you're always going to have the corporate consolidation where everything just kind of eventually becomes part of McMicroBucksmart. And they're like the only corp and they own everything. And that's the natural endpoint of the capitalist society that they've built.

It's up to us to try and keep our packages kind of like out of that system. And it's going to be up to the authors of those content packages to stop using things like NPM if they get too corporate or if they start restricting access to people with paid membership. So they stop doing open source. They're going to have to go somewhere else and make a new system. You two, I have you on usually to counterbalance me. Like when I go off talking on this rant about something, then you guys, you know, be like, why is that important? Well, what side am I going to take? The side of like corporate?

It's like, oh no, everything should be like proprietary. That's the problem today though, right? I don't have you two to counterbalance me. You guys are both like, yeah, no, submit to socialism. It's the way forward. They're like, no, I actually, I'm not against capitalism at all. Like, I just think that the package world, like this is something cool.

We shouldn't let this fade away. This like community of people building awesome things. But do you think it's fading though? Because like I see open source communities all over the place. Oh, I don't think it's fading at all. But like, I worry, like the bigger you get, the harder it is to keep a community intact. I see backlash of people against open source users. Like there was recently this Python guy, this guy made a Python package and it was considered critical because so many people used it. And they asked him to set up two factor authentication to post updates. And he was like, fuck you. And he deleted it.

And he deleted it. Yeah. What's happened is that guy doesn't feel like a part of that community. He feels like they're taking from him. He's having to maintain the software and he's not getting anything back. Right. He's not feeling the love from that community. But isn't that a little unreasonable though? Like put two factor in your thing and it's like, fuck you. And he like deletes it. Don't you think that that's a little extreme? Oh no, I do. I mean, some people are just jerks, too. I mean, some people make things, make packages because it's for themselves.

That's the con of the community, right? The community is full of people and people are flawed. So you're going to have people that are just like, meh, and they delete their package. Oh, well, whoops. So I'll tell you what I think my solution is.

The original CPAM was built by a community. It was built by a community for a community. And that's the way we fix things. You just have to lean in to the community aspect. And this is like a super vague plan to save the world of packages that I just kind of made up. But like we celebrate package creators.

Like in the CPAN world, one thing they had was like module advents, like an advent calendar, where for the month of December leading up to Christmas, people would like share cool packages, you know, like on their blog or whatever. And this is spread to other communities. And I think celebrating the authors of these packages is super important. People are often not doing this for the money, or if they are, the amount of money that like you could give them probably wouldn't make a difference anyway. They're often doing it because of this ownership

as part of this community, right? Package repositories and open source, they seem like code, but they're not really about code, right? They're really communities. They're like groups of people and you need to nurture that, like treat them as people. And when I say you, I mean like specifically, like Dawn, Crystal, me, Adam, the listeners.

We need to work to make the community better. People are hard. People need care. If you build a community and it's successful, it'll keep growing and it'll get bigger. And then you'll start hitting some of these problems like NPM. So people shouldn't laugh at NPM because of these problems. They're just actually so big that it's hard to keep the community together. This is more of a policy recommendation

at some size, like communities need to have the community's interest at heart. They should have like some sort of board of people who make recommendations against it. Their criteria should be what benefits the community. Cause this is kind of like a weird collectivist, communist-esque thing, right? You sound like you want to create a union. That's what you sound like. You need a union of package creators so that they have a collective voice against. But it's not a voice against. Let me give you an example. So if I create a package and then I just lose interest in the community and people are like, there needs to be updates in that. Like right now it's not clear, like I own the package, right? And maybe I don't want to open it. Maybe I'll just like delete it

or whatever. Right. But a community kind of needs to protect the community's interest. At some point, if something's important enough, it's more important than that single author. People should be able to step in to make changes to it, to improve it. It's putting the community's interest at a certain point above the individual contributors. Like you need to celebrate the contributors, but at some point you need to protect the community, which probably is what NPM was trying to do by undeleting that package, right? Imagine if you, yeah, if you said like, listen, when you push your package to whatever, Adam's repo, like you agree that, you know, there's a process for taking it over. Like if we deem it to be important enough, like we'll take it over. Too many people depend

on it. It's like, yeah, the, the CNCF, which is big in the Kubernetes world and the Apache foundation, they both kind of work this way. You donate important software to them and you can still help, you know, make it work open source and whatever, but they have processes in place to benefit the users of it. Because that's how you build something collectivist. You kind of keep the collectivist ideas in mind. If you build up goodwill, if you try to have the community's interest at heart and everybody tries to get along to the extent that they can. And then occasionally you take actions

taking a package from somebody. I think the community will have your back because they know that you have their best interests at heart. Let's not go back to the world of commercial packages being handed down to us morts to consume from on high, right? Pearl has fallen out of vogue. Maybe the reasons that it happened are valid. Maybe they're not.

But everybody's learned how CPAN works and this idea has spread. You know, CPAN's releases are actually dwindling now, but they're actually far above where they were back in 2004 because just the world of software is so much bigger now. And this is when it gets hard, right?

I think that there's going to be more and more stories about like package stuff going awry and people are going to say like, this is a mess and we need to do X and Y. But I think they're wrong. Like, I think the thing we need to do is just come together and support each other and build cool software. If you want to live in this world of open source communities, probably you don't maintain some major open source package yourself. Like statistically, most people are just consuming this stuff, not generating it. But you use it and you do issues and you ask for features.

And so nurture it. Thank the creators, follow them online, help them with their package, stick money in their pocket, buy them a coffee. They're the goose that lays the golden egg that is our world of software development. That was my long rant. That was true.

That was the show. Thank you, Don and Crystal, for being here. Thank you to Andreas and Jarko and Tim Bunsen, everyone who made CPAN what it is. Everybody who uploaded things and helped build that amazing community. And thanks also to the online CPAN timeline for laying out a lot of these details for me.

If you like the show, please support me on Patreon for access to tons of bonus content. And until next time, thank you so much for listening.