The Changelog: Software Development, Open Source - Keep npm Running (Interview)

Episode Date: November 26, 2013

Isaac Schlueter and Charlie Robbins joined the show to talk about the "crashyness" of npm recently and the community fundraiser they are starting to ask the community to support npm and to keep it running. Isaac is the creator of npm and a maintainer of Node.js. Charlie is the co-founder and CEO of Nodejitsu.

Welcome back, everyone. This is The Change Log, where a member-supported blog, podcast, and weekly email that covers what's fresh and what's new in open source. Check out the blog at thechangelog.com, our past shows at 5by5.tv slash changelog, and subscribe to The Change Log Weekly. It's our weekly shows at 5by5.tv slash changelog, and subscribe to the Changelog Weekly. It's our weekly email covering everything that hits our open source radar. We ship it on Saturdays.

You don't want to miss it. Subscribe at thechangelog.com slash weekly. And this is episode 113, and today's show is sponsored by DigitalOcean and TopTal. We'll tell you a bit more about TopTal here in just a bit later in the show, but Toptal connects startups, businesses, and organizations to a growing network of elite engineers around the world. Check them out at Toptal, T-O-P-T-A-L.com. And DigitalOcean has been supporting us for quite a while.

We love DigitalOcean. We're actually hosted on DigitalOcean right now, which I have to say is fantastic. And we want you to get hosted on blazing fast digital SSD cloud servers today. They're a simple cloud hosting provider built for developers. You can easily create a new droplet, which is basically a server.

And you can get root access in 55 seconds. Literally, in 55 seconds, you will be shelled into your new machine. You get a choice of size. So you want a large server, a small server, a lot of RAM, not a lot of RAM. Choose a region, whether it's New York or Amsterdam or one of their other locations where they have data centers at. And also the flavor of OS. So no matter what it is, Arch Linux, Ubuntu, whatever your choice is, within 55 seconds you'll have all that set up and you'll literally be SSH'd into that machine all through an easy to use dashboard.

You can enable backups, take snapshots to your server, resize up or down as needed. And they are, and when I say they, DigitalOcean is optimized for developer user experience. I cannot stretch that enough, that it is so easy and so pleasant to use. You've got to try it. Try DigitalOcean today for free. Use the promo code we have, changelogsentme. That's changelogsentme. When you sign up, you're going to enter your credit card information, and near that spot there, there's a spot where you can add that coupon code.

Go ahead and pop it in there, which will give you a $10 hosting credit or two months free if you're going with the lowest plan. Head to DigitalOcean.com right now to get started telling the change law who sent you. And we're joined by Charlie Robbins and Isaac Schluter. Charlie is the co-founder and CEO of NodeJitsu. And Isaac, you're back again. You're the creator of NPM and the maintainer of Node.js. And I got to welcome Charlie. You're the first time on the show, but Isaac, you're back again. You're the creator of NPM and the maintainer of Node.js. And I got to welcome – Charlie, you're the first time on the show, but Isaac, you're back again. You were recently on, I think it was 101, I believe is what it was.

Sounds about right. Sounds about right, yeah. We had you talking about, I guess, a little bit of burnout, a little bit of NPM history, some origins. And I guess part of this call here is kind of going deeper into the origins of NPM and mostly around the registry. But before we dive deep into the show, let's do a quick round of introductions. Charlie, why don't you introduce yourself real quick, bud? How's it going, everyone? I'm Charlie Robbins. I'm the co-founder and CEO of NodeJS. In addition to running a platform as a service for Node.js, we also host the public

NPM registry, which is why I'm here today to talk to you about that. And we also have an enterprise private NPM product to help your organization work better with NPM and Node. Awesome. And Isaac, I guess, do we have to announce who you are? Everybody might know, right? Well, just to be on the safe side and because my head because my head is big, you know, it keeps me, keeps my ego inflated. I'm Isaac. I wrote NPM and I work on Node. I am currently a joint employee and been on the internet since there was an internet. And if you want to, for the listeners of this show who may not have listened to episode

101, Isaac was on that show. We talked about the origins of NPM and some different details around that and great show. Early days of NPM stuff. Yeah, yeah, exactly. So I would definitely encourage you either before or after this show. I'm not sure what matters really to you, but go back and listen to that show for sure. But this is episode 113, so we're 12 episodes past that. And I guess a bit has gone on since then too.

I mean that was – let me check my details here. So this – the date we did that show, Isaac, may be around the time that you guys have started to hear some of the details of what we're going to discuss in the show. And the main premise of this show, I guess it was kind of going to be, for lack of better terms, like a public service announcement. Like something's going on with the registry. Y'all got something cool happening. You're announcing some news tomorrow. Today's Monday, by the way. If you're listening to this on Tuesday, you probably heard some big news and you're jumping

on board to help out. So I guess which one of you want to tee off what's going on? I guess I'll take the lead there because that has been what's keeping me up at night the last few weeks. Charlie, right? Because you guys sound a little similar. Yeah. Yeah, this is Charlie. OK, cool. Charlie, you should – just to help distinguish, you should talk in an exaggerated New Yorker accent. Hey, I'm walking here. I'm going to talk about programming.

That's right because you're from the East Coast, right? Yeah, that's right. Well, so is Isaac. From Connecticut, so it's a little different. Not very, though. Basically the same place for all you California people. Absolutely. Kind of.

New Jersey and Connecticut are the armpits to New York's brain. Apologies to everyone from New Jersey and Connecticut, but I'm from New York. It's my obligation to make fun of you. We're in a fight now. Yeah, so getting to the serious things. Yeah, the registry has been having some stability issues over the last month. And again, this is Monday, but Tuesday, there will be a postmortem. And so at this point, if you're listening, you've probably already read it. And we've put a lot of time and effort into figuring out what root cause was. And we haven't quite gotten there, but we do know the solution, which is it needs more resources.

And we actually have been hosting the registry for about the last six months since we acquired Iris Couch, and they've been hosting it forever. And Isaac will probably talk a little bit more about that later. But what we need right now is more resources. And we do this completely as a community service. We don't take any money from anyone for it. And what we're looking to do today is we're starting a crowdfunding campaign for NPM so that we can keep it running and keep it awesome. And so we're asking individuals and organizations to reach out. The website is scalennpm.org.

You can check it out now. And I think that's officially the end of my shtick. So I'll pass it back to you, Adam. Well, I want to touch on one thing. Maybe we can go deeper into this, but you'd mentioned organizations as well as people can kind of play a part in this. I noticed that I've got early access to this, I guess, non-public version yet since it's Monday, not Tuesday yet. So I see some logos there. Are those real logos or are those fake for now? No, those are real logos. Those people have committed, and we're excited to have them as launch partners.

We are waiting to get the final acts from one of them. So I'm going to hold off the talk tomorrow. So anybody hearing will be like, oh, what are they talking about? That's on the website. And when we talk about this crowdfunding, this isn't new, right? I mean, this is something that, I mean, there's Kickstarter. So this isn't new right I mean this is something that I mean there's kickstarter so this isn't like new things but when Andrew and I talked a little bit earlier about this because you know Andrew if y'all noticed that Andrew's not on the show he couldn't make it today so I had to fly a

solo myself but we talked quickly about this and he was like this is really neat I mean doing this I mean getting the community behind NPM and even like you're branding around like you are npm and that's the community right like you're doing this as a public service you know you charlie and no jitsu and the service you're doing there and isaac obviously you wrote it and have been working really hard to lift up the community but you know it's really a community thing like there's so much new stuff happening that's distributed through NPM. It's crazy. I mean, and that's kind of why you're in this boat today. Yeah, file this under good problems, right, Isaac? I don't actually think there are any such thing as good problems.

It's a problem that indicates that we're doing something right, but it's a problem because we're not doing everything right. And the bottom line is we need more resources to make the NPM registry stay up and stay good. So that has to come from somewhere. So we're asking people to help out. Can we rewind a bit, though, and maybe kind of talk about the early days of the NPM registry and what that was like and kind of how we got to maybe a year ago? A very simple version of what's up there right now was actually written by Michael Rogers when he worked at Couch.io, which later became Couch.one and then merged with Membase and became Couchbase. When that happened, and so when they were Couch.one, I believe, they took on hosting of the NPM registry just as a community service. And that was when Iris Couch spun off from CouchOne under being run by Jason Smith and Jeff Jackson. They continued to run the NPM registry

and actually developed an Iris NPM product. And kind of the handshake agreement was, you keep running the registry and keep not charging me, and selling NPM registries is the way that can potentially fund this. And that's kind of where we left it. After a bit of time, they merged with Nojitsu or purchased by Nojitsu. Iris Couch was purchased by Nojitsu. So now Nojitsu is selling on-prem NPM registry clones and also providing hosting for the public NPM registry as a community service. The interesting thing is that this year, in 2013, we've experienced 10x growth in most metrics,

and in fact more than that in some. The number of downloads per month just is one rough metric of activity and size. The number of downloads per month has gone from about 13 million per month to well over 100 million per month. So basically we're at this point where the money to continue providing this hosting is not growing quite as quickly as the costs of providing this hosting and so the the thought then i mean if i'm reading through the details right it sounds like uh charlie on your side to i mean obviously you want to do something good you're part of the

you're obviously invested in node right i mean that's clear with the business name what you do is that's evident right but to fund that, you've got like a private enterprise on-premise NPM. That's one thing. And then you've also got NPM registry that you can also install. Is that right? So it is an enterprise product that provides some features that you don't really get with even a vanilla clone of the public registry. So one of the things that we get as the host of the public registry is we have all of the hashed passwords. Obviously, we're never going to release those,

and they're kept in a very secure manner. But one thing we can do is replicate specific ones of those to our customers' private NPMs. So that means that your company, when you tell someone to point to a new registry, they don't have to sign up again. They don't have to go through that experience. They can use the same NPM credentials that they have for the public registry on their private registry. Smart. Yeah.

And in addition to that, there's some additional policy-based things where you only want to have a subset of the registry or you want to know which packages are yours that are private. Those sorts of higher level, large organization problems. And so I guess if I'm understanding this right, Isaac, you'd mentioned that that part of it isn't quite moving as fast enough to keep up with the demands to, one, labor servers, just in general, the resources necessary to keep the NPM registry to its maximum potential, keeping up time right. We've kind of gotten to this point now where we don't have enough man hours and enough servers to kind of go around to make it as responsive as it needs to be. Right, exactly. And in addition to that, what also comes along with 10x growth, in addition to additional resource utilization and so on, is that there's more people depending on it.

So a smaller hiccup of service has a much bigger impact on a much bigger number of people. Whereas previously, if the NPM registry did go down for a few minutes, chances are nobody would get bothered with it because, you know, that's just like statistics. Whereas now, I mean, even relatively minor outages end up impacting a lot of people, causing their builds to break. And, you know, there's a lot of impact from that now. Yeah. I've been noticing on the NPM.js Twitter handle too. Is it you that run that or is it you, Charlie, you, Isaac, and one other person? Isn't that right? The NPM Twitter handle is NPM. Yeah, I mean it's self-aware.

We just service it. Yeah, we're just helping it out. I mean that's – I don't really know. I don't really understand your question. Oh, like who is behind it? Because what I was going to say was that I've seen lately, you know, kind of responding back to certain people and saying, okay, you've got an issue here. There's a package here. We're working on that. And there's issues being obviously filed against the GitHub issue database. And so you're playing, I guess, triage and support to the community through Twitter. And there's, you know, over the last few weeks, you've had some scenarios where you've had

to, you know, kind of look at what's going on and apply some fixes. I was trying to figure out who runs it. I was trying to key off that, basically. Oh, yeah, yeah. I guess the... I wasn't sure who was doing the talking, basically. I was being slightly tongue-in-cheek. The NPM Twitter account is the character of NPM. NPM loves you.

Much more than any of us do, that's for damn sure. I see, I see. I'm not really sure where to go with that one. It's kind of like when my mom was Santa Claus. Like she – Right. You know, she asserted that Santa Claus really existed because he's an important part of her psyche. Right, right.

Gotcha, gotcha. So we've got a crowdfunding goal of $200,000. I guess you probably expect this question, but what does the money get used for since we're talking about resources? So we expect the server bills for NPM right now are about $10,000 a month. And that's actually probably going to grow a little bit as we start to move off some additional things. Because on those servers, we just really can't do anything that's IO intensive anymore. For example, when we tweet out the statistics of NPM every month, those used to be crunched from log files that sit on those servers. On those servers, can't do that anymore.

We need to have a separate log server that all that I.O. gets done on. We need to have a hot spare that's in continuous replication, not just in case there's a crash, but in case the disk size continues to grow the way that it's been growing over the last few weeks. So we have to regularly start running compaction on CouchDB to keep that disk size down because from a CouchDB perspective, and this is sort of an interesting story, the outage on November 13th where we actually switched over to Multimaster, Jason and I were in Vancouver to go to CouchConf. So we woke up to go to a CouchDB conference with CouchDB falling over. And with a room of catch db core committers nobody really knew what was going on because the attachments were never really meant to be 99.97 percent of all

bytes in the database which is basically what you get in npm so we just run into these really strange scenarios that um you just don't see anywhere else. So we can throw more hardware at it for now, and it works just fine. And then Isaac is leading the charge on refactoring things to get those out of the registry and into a CDN somewhere. And you mentioned, I guess, a bit earlier that your hope is that this fundraiser is more of like a shim for now. It's not a long-term solution. It's more of like a here's how the community can help give back to keeping NPM up or keeping the help facilitate that through, you know,

node being able to, or node being able to get that product out there and be sustainable to supply the needed funds to run it. Absolutely. We've done some sales of our private NPM products so far. That's really starting to scale up. And I think by this time next year, it's going to be a completely sustainable business.

And we'll be able to provide this service to the community the way that we've been doing up until now. What type of, I guess, who is it that uses, what kind of scenario can you best paint that would use this private enterprise NPM registry? I like to paint it in the way that we used it at Nojitsu because we've had the most, all of the NPM use cases, we've had them. So all of our stack is Node. So we want to encourage this same idea of modularity and innovation through modularity in the organization

so that one of our engineers can say, I want to write a module to do X that's sort of internal, something that wraps our API in some unique way. They can name it, publish it to the private NPM, and feel that same sense of ownership that they have in their open source code. At the same time, we also run a platform as a service product that runs in a different data center than the NPM registry. I would say that basically all platform-as-a-service products run in a different data center than the NPM registry, which is SoftLayer US East.

And so we run a full replica of the public registry in Joyent's US East data center to remove downtime in case the public registry does go down, and then also to reduce latency because all of those package gets are going over the local intranet and not going out to the public internet. So any gamut of those things there could be helpful. So if you're having to use Node, be you Rackspace or Joint or Heroku or any of those types of companies running a public replica

in the same way that they run apt or yum replicas is super valuable. And for private organizations, it's a way to scale and distribute the workload of your Node.js code base in an organic way throughout your organization,

the same way that it's distributed organically throughout the Node community itself. So I guess with the enterprise pieces, this is assuming that a lot more people in the enterprise are picking up Node and using it in ways that their organizations are going to use this and want to publish private packages to be able to serve. And that's kind of hinged on that fact, too. Is that growth still going the direction you guys want to see it go? Yeah. I mean, that's definitely the way that things are going. I think that if you look at how NPM is being picked up and how Node is being picked up at companies like Yahoo and Walmart, they are using it internally to manage their

dependencies. And this kind of enterprise product makes it a lot easier and more accessible for more companies to do that. Yahoo will probably just hire the people that they need to manage that in-house, but they're gigantic. And there's a lot of companies that are a little bit smaller or even if they are of the same size, they have a little bit less of a DevOps culture. And I think, for example, Walmart is a perfect example of that.

They have a lot of technical work that they're doing in Node. They have several teams that want to share code and interoperate. And NPM makes that extremely easy to do so as long as they can remove themselves from the um you know from any impact of the public registry having any problems it makes a lot of sense for them on that you know from like a safety net point of view but also they want to be able to publish code that you know just published to other teams inside their own firewall and be very strict about which programs they allow their developers to pull in. So for example, they can have license auditing or even security reviews and so forth. And that's not something that we're likely to add anytime soon to the public registry.

So having that, but it's a big feature for enterprises who are using NPM internally. You know, we had this weekly email we sent out and just talking about Walmart, we linked out to something on the joint. I think it's their, I think it's the blog, but it was a video. And it was this fellow named Aaron Hammer. Yeah. How do you say it? Aaron. Aaron. Okay. Aaron. Yeah. How do you say it? Aaron. Aaron. Okay. Aaron. Yeah. Yeah.

And I was like – my eyes are glazing over it and obviously at some points, but I'm like the billion-dollar question, which is how not Walmarts and not Yahoos? Or are there just a lot more Walmarts and Yahoos we don't know about that will utilize an enterprise system that will extremely data intensive and and you know need to do a lot of io and kind of be this sort of central hub middle layer um and you know condi nast is uh is using it the wall street journal the uh new york times um you know lots and lots of them if you go go to nodejs.org slash industry, that's a pretty small subset, in fact, of the companies that are using Node in a really big way. And there's some really well-known names there on that list. Basically, those are just the subset that have noticed this page and decided to send me a pull rec to put them on it. So there's a step to get on this page,

and we have honestly no clue how many people are using Node or exactly what they're using it for, just because that's the nature of open source. But being at Joyent and getting involved in some production issues and things that come up with our customers. I mean, yeah, Node is very big at a lot of companies that are of the similar size to Walmart. I think Walmart's probably one of the biggest – they're one of the biggest companies, period, right? But there's a lot of companies that have a need forPM services and, and have the money to pay for it.

So I guess this has been a kind of maybe a 12 minute rant or so on whether or not, I guess the core crux of the question was, you know, do you guys both truly believe that? And obviously, you know, no Jitsu and Charlie, you do because you've, you're building this product, but you know, is, is the private, you do because you're building this product. But, you know, is the private side of this going to be able in the future to sustain it? I mean, so that's the goal. And how you think it's a year away from that or what are the challenges to get there?

I think we actually could be as close as six months away from that. The challenge for us is really streamlining the process here. And right now, the big blocker for us is that the registry is quite large. and your credit card and here's an NPM registry for you without expending a large amount of resources because we need to copy over roughly a hundred gigabytes to a new server when that happens. And so that's a function of really the, the disk IO that couch DB needs. We can't put that on say an EBS volume or a some sort of network storage of some kind because that is just not fast enough. Couch sort of tends to get behind itself or ahead of itself with these reads and writes when the disk I.O. is not fast enough.

So we have a process now where someone can sign up and we can get something provisioned within 48 generally 24 hours but making that easier and getting those those sales done faster is our main focus right now on that product well let's pause for a minute and give a shout out to our sponsor top towel they'll be sponsoring the show for another month so good news there and certainly appreciate the support of top towel to the show. I've been talking to Brendan, their co-founder and CTO, and I kind of mentioned before that I wasn't quite sure what to expect. But since then, Brendan and I have had a number of conversations, and he's kind of really helped me understand what their mission is. And I've got to say these guys are the real deal. They're engineers themselves.

They built the entire company around engineering from top to bottom. They're not non-technical recruiters trying to pimp developers, for lack of better terms. They're a network of elite engineers from all around the world who work with some really awesome clients. And for those of you out there who are freelancing or like to test out freelancing, you got to check out TopTow. You can work on special projects with companies like Airbnb, Artsy, IDEO, and many others. You can work remotely or on a beach, which is always fun, or anywhere in the world. And to get started, you've just got to go to toptow.com slash developer and click join the best. And because they want to work with only the best team engineers out there,

they've got a well-thought-out four-stage screening process that they use that begins with a personal Skype conversation. They get to know who you are. They introduce you to TopTal and kind of help you understand what their mission is and see if you're a fit. And from end to end, the entire screening process includes an English

speaking test, a timed algorithm test, technical interviews with core TopTile engineers, as well as a test project. And once you've made it through the screening process, the sky is the limit. And if you think you have what it takes, head to TopTile.com slash developer to get started. Tell them the changelog sent you. Isaac, I know you kind of touched a little bit earlier on like the early versions of

it, but can you kind of give me me for those out there who would totally be interested in this part? I certainly am. But, uh, how does NPM currently work? Like what is the current setup? And when we get these funds and this crowdfunding is successful,

which it's going to be, because I know you guys are awesome. So this is, community is going to love this, but totally going to support it. But, you know, when we get to the next version of it,

how, how's it work now? And how's it going to going to work when we get fully funded for this fundraiser? So in a nutshell, the NPM registry is a couch DB with a little bit of rewrite action kind of at certain like shows and views and such. When you publish a package that's doing a put into the CacheDB and there's a bunch of rules that, you know, make sure that it's following a few basic guidelines and whatnot and not doing anything insecure, set up in the validate doc update function. There's also the actual tarball, which contains the contents of the package, and then that's added as an attachment on the document. So there's one document per package,

which has like a versions object that has the individual package.json data for each published version, and then also has a tarball as an attachment. So the problem is that CouchDB is good at handling attachments, but it's not great at handling as much attachment load as we've put into it. And we've kind of reached well past the breaking point of what this database is actually good for.

What it's great for is storing, um, Jason blobs and doing map reduce over them. Like couch CB actually totally, totally is great for that. And, and you know, they also have like really nice restful APIs really,

which is obviously a big win when you're, um, you know, when you're, when you're doing stuff with node, which, you know, NPM is

just a Node program. So, what the plan is, is one thing that I've been kind of working on as a sort of side project, thinking about and not really gotten too serious about until relatively recently, is this project of taking all of the attachments

out, excuse me, and putting them into Joyent's cloud hosting service called Manta. So this gives us a number of benefits. First and foremost, if we have all of the attachments in one place, it's very easy to make that the origin server for a CDN network. I have an offer from MaxCDN to provide free CDN services in exchange for a little bit of link love and so on. And so that's going to be really awesome.

But in order to do that, we need to get everything in as the, behind the single origin server URL. So we can say, okay, map this path to this path in the CDN. So I've been working on the process to make sure that we can get things out of there. But what we can't do is we can't go through

and replace everybody's NPM client overnight. So any changes that we make to the actual client application have to be done and then published with a uh with a node release and then you know we need to sit on it for like six months wait for the request to the old url to kind of taper off it's just like very very long process right so uh what i've been trying to figure out how to do is basically how to move forward with this without um without breaking backwards compatibility at our our api layer so um what we've done is or what i'm planning on doing is once i get everything moved into manta there's already kind of a first pass of this when we uh banged on it a little bit

and found a few problems and kind of circling back and updating some of that stuff. Once that's in place, the URL in the metadata of the CouchDB that tells the NPM client where to go download the tarball from, basically each time it gets an update from Couch, it's going to take that tarball, put it into Manta, where it's behind update from Couch, it's going to take that tarball, put it into Manta, where it's behind the CDN, and then change the URL to point to the CDN URL rather than the direct CouchDB URL. Once we do that, then there's a couple of options that we have. Newer NPM clients already know how to interpret this. There's a, I don't know if you want to call it a bug. It was actually an early

workaround for a bug that no longer exists. But like, you know, that's how it goes with code. There is a shortcoming of the previous versions of NPM client where it will always try to fetch the tarball from the same registry host, no matter what. So we need to do some other magic. And we've kind of explored different ways that we can either modify CouchDB or take some liberties with the way that the NPM registry Couch app works such that it will still pull those attachments from the CDN rather than from an attachment URL on the CouchDB. Once we're at that point, we can actually start removing those attachments altogether.

And even before we remove them altogether, as long as those requests aren't coming in for them, it'll be a lot easier because there won't be as much disk IO. And it seems like it's a lot of orchestration around this. I know that, I mean, it's nothing to compare it to, but just when you move a site from one server to another, there's a lot of orchestration around that. And this is like that times a million, right?

Like it's, you know, everybody banging on NPM, you know, either installing or deploying or, you know, pushing up their own packages. So how does the community, I guess, how, I mean, is this something that you need to orchestrate in some sort of like syncopated manner? How does the world fall in place to your plan here? Well, I think basically everything that we're planning on doing, we can do with

little or no downtime. I mean, with something like NPM, if we do need to have some kind of downtime to restart a server or change the way things operate, you really need to make sure that that counts. And so you kind of want to plan everything that you need to do and get it ready and then minimize the downtime so you can be back up and serving requests right away. In this case, I don't think we'll even need that because of just the nature of the way that Couch operates. We already have two replicas that are in continual peer-to-peer replication with one another and then a load balancer in front of them.

So, you know, we can start operating on one of them, either take it out of rotation and then do the thing and put it back in and so on. I mean, there shouldn't be any interruption of service throughout all of this. And in fact, most NPM users won't even notice that anything happens. Once the CDN starts being the target for all of those tarball downloads, especially users in Southeast Asia and Australia will notice that things get quite a bit faster. But otherwise, for the most part, it should be only increases in stability as we move forward. Just because I'm trying to really stay on point with this one, but when you say clients,

what you mean by that is like me at my computer, either installing or pulling from NPM, right? Yep. Okay. I mean, any time you type NPM whatever on your command line and it has to go to the registry. So that's, I mean, that's a lot of applications. That's a lot of different commands, but mostly you're either, you're either downloading metadata and looking at it, you're downloading tarballs and installing them or you're pushing stuff up

to the registry. So yeah, all of those operations, whenever I say NPM client, I mean like the, the program called NPM. Right, right. And so with that, you'd mentioned an update that has to happen for the client. So I guess those who may not go and pull down the latest version of it, whenever you kind of start to orchestrate this plan here, I mean, is it – how do you – I guess if I don't upgrade or update my NPM, what happens to me? Well, I mean, for at least until everybody else also moves on, you should be fine. I have a very strong feeling that when people are using your program in production, it's kind of a dick move to break it. And that dramatically slows us down sometimes. But on the plus side, it means that things keep working for people and that they don't ever really notice. So what we do is we make whatever change we need to make in the

client. I usually set like a six month reminder on my calendar to revisit the issue. And then, you know, we take a look at it and see if we're still getting requests to that old URL or what have you. And if we can tell that it's, you's a very, very small percentage of users who won't be impacted, we might just go ahead and make the change and, you know, okay, a couple people have to upgrade. But as long as they're at that point, they have had ample opportunity. So it's not such a jerk thing. It's their own fault, right? Move on, right? Yeah. Gotcha. Well, I wouldn't say it's their fault.

But it's reasonable to expect that if they haven't upgraded, at least it's easy enough for them to upgrade by this point. You know, there's, there is a version of NPM that works with their version of no, that has access to this new thing. And, and so it's not an issue. We obviously see some of the reasons why it's important, but I want to hear from you guys, you know, what, what is it that's important why the community steps up to support? It's like, you got this branding around that, like it's your NPM, you know what what is it that's important why the community steps up to support it's like you got this branding around like it's your npm you know you want to keep it up you want to keep

it fast you know what is what is the the importance i guess of the community stepping up to help support this effort of keeping npm running um i'll jump in on there on that um isaac and i have actually talked about this a lot because the is obviously very excited about this migration to Manta, as am I. But from a standpoint of us as a company, that's actually a lot of long ball labor costs that are hard to ballpark. And it turns out actually the person who's most suited to do this work on the CouchDB side is Jason, our CTO. And so from our perspective, we have this thing that we run, which we're really happy to run, but we also have this product that we're building that also needs Jason's time. And so if we're going to prioritize his time to make that the

thing that needs to get done and takes priority over our product, we need to subsidize that in some way going forward. And that's where we say our costs are doubling, not just servers, but labor to take us to the next order of magnitude. So let's talk about the actual fundraiser itself. We talked a little bit earlier about the goal that you have set. And like any crowdfunding, you've got many levels and you've got the opportunity for not only individuals but also companies to take part in this. And you've got a couple that are on the site now, which we can't mention because we're not really sure if at least one of them is.

So I'll just leave them both out the gate. But talk about how – I guess maybe the last – when did this idea come about to do a fundraiser to make this ever possible? Not just so much asking the community for their support but actually turning it into a crowdfunding with these levels and what you guys are doing with it. So the person who actually suggested this to me was – it was actually at CouchDBConf. What was that? Ten days ago. I was on a call with Nuno Job. Congratulations, Nuno and Paula.

They just had a baby. He was talking with me, and we were talking about how the downtime was just taking up. It had literally sapped our whole week. He had just said, look, you should do something like what Travis did for their crowdfunding campaign. So Travis CI ran a successful crowdfunding campaign in 2012, I believe, or maybe it was through 2013, called the love.travisci.org. And the parallels were really obvious there. You have this thing that is deeply integrated into the community that people rely on

and is also on its way to becoming a sustainable product. But we need help to get there, just like they did. And I think they've done a lot with the money that they raised last year. And so with those parallels in place, it just became obvious that this is what we need to do. And so we moved very, very quickly to get this out the door because we didn't want to lose the, or didn't want people to forget the pain that they felt when this happened. Because it's very easy with a service that you depend on to be mad at it when it's down and then just forget about that later on and then just be mad again later when it goes down again without really thinking about, okay, well, why did it go down in the first place? Is that a symptom of a greater problem?

Can that be prevented? and not in the way that you generally probably read postmortems for the services that you use, but in a holistic community way because this service, again, is not for profit in any way. I'm going to read a tweet I saw actually from Sven Lido. He's a hacker at Hoodie, which was recently on the show, and also on Bower. And something he had tweeted, I don't know if it was actually from him or not. Maybe it was an overheard, who knows? But he said, uh, as a developer, I want super fast NPM, everyone always. So everyone wants it fast, right? They want it, they want it fast. They want it now. So that's, that tends to be that. And I think, you know, Charlie, you and I talked a couple of days

ago, kind of prepping for this call. And I was just thinking like, you know, as somebody in open source, right, you just expect the service to be there, but you forget what's behind it all. Like you guys just talked about this entire re-architecture that involves brand new, cool, blazing awesome stuff from Joyent and MaxCDN giving their support and all these different things you're having to do to orchestrate this stuff. And those who are using Grunt or Bower, all these newer things that are kind of front-end tools that maybe they're not used to what a registry might be because some front-end developers are kind of getting into using something like RubyGems would be. That might be newer to them. They just think, oh, it's a service. It'll be there.

But all the while, you guys as a business have to support this thing, and you, Isaac, have to work really hard to deviate and kind of coordinate things for the community to keep NPM running well and fast. It's tough. I mean, this is probably proof of that, right? Well, you know, big things are fun, too. Yeah. Yeah, right? I mean, if you're going to have a hard know how important it is to mention some of the different goals you guys have.

But the entire overall goal is $200,000 you're trying to raise. The campaign is in 30 days. What happens, I guess, with the traditional crowdfunding? Let's say you don't fund the full 230 days. What are some of the takeaways or changes that – is it all or nothing? How is this crowdfunding goal a little different than maybe others might have been? So we opted to go outside of say Kickstarter or Indiegogo because if we got 180 or 100, that's still going to get us further along than we are now. And this is such an important public utility that we didn't want to be an all or nothing place, which is why this actually runs through, this

is a totally custom site that we've built at Nojitzer over the last week, sort of coincided with a new version of our billing system, which makes doing this very, very easy. And considering that we're going to launch with a pretty significant portion of this already committed from companies, I'm feeling confident that we'll hit the goal. But that is obviously always a concern. And from our perspective, if we don't get there, we're going to do as much as we can with the money that we get. So the registry costs will be subsidized that way. The other important thing to realize is that we're also a company and that this, when you play that, and this probably doesn't often get talked about on a show about open source. You talk about signaling when you run a company. And this, even just the support that we've gotten now, is a very positive signal for what we're doing. And I think that that's going to be reflected in how our business operates and raises capital over the next six

months. And that's a big impetus for this is to really demonstrate to the community, the larger community, the larger investment community, the larger software community, that there is something special going on here. And it's not just a lot of hand-waving and China market internet scale words getting thrown around. Since we're throwing around a couple words, I was thinking about something as I was driving around thinking about this call earlier in my day today and I was thinking if I had to tell somebody something, I'd just say, put your money where you put your packages. I'm not sure if that's accurate or not.

You didn't say it. It's not your marketing thing, but I was thinking that's pretty accurate. What do you think? Put your money where you put your packages i'm not sure if that's accurate or not you didn't say it's not your marketing thing but i was thinking that's pretty accurate what do you think put your money where you put your packages yeah that's that's a that's a good uh good slogan all right cool uh awesome you know please please don't put your money in npm literally don't don't publish bitcoins yeah don't do this it is not secure for that. We are not a bank. One thing that was on this page too, and I want to just point this out to those that are maybe at the page right now. Just to kind of recap on the URL this is going to be in. I want to ask you one question about this too once I mention this.

But the URL is scalenpm.org. But on that page, about halfway down where it says why is this important um i've been on this page i don't know how long maybe a half hour i think or a couple hours i don't know maybe i've had it sitting here that's i think it's like an hour and since that hour's passed there's been over a quarter million uh packages. Like, this thing is, like, it's on fire. It's crazy. Yeah, and that's one of the things that, you know, is really nice about the data statistics that we get

is that we can go out and crunch that data and infer it that way. So what's behind the, I mean, obviously a URL is a URL, but what's the significance of scale NPM? I mean, obviously you're scaling it, but why not just opt for a subdomain or something like that, like love.travisci.org.com. I think it's.org for their open source. No comment.

Okay. Gotcha. Let's see. I think that's pretty much all i wanted to ask you guys i guess about about what uh what's happening here i think it's just pretty neat that um that you're doing this i think it's i mean anytime you get a chance to involve the community i know they the community always ends up uh you know, being excited about what you're doing for, but then also just appreciating the fact that you let them take part. So even if it's $5 or $100 or whatever it is, whatever you can afford to support this, we definitely would love you to do that. That's why Isaac's on the show. That's why Charlie's on the show.

That's why they're working really hard for this. So go to scalenpm.org to check that out and give your support. But a couple of traditional questions we ask on the show, which I don't think it's a problem here. And Isaac, I know you asked or answered a couple before, but if you weren't trying, I guess I'll ask this question for you, which is if you weren't, I guess, on this call with me right now and Isaac talking about this and you weren't building NodeJitsu, what would you be doing?

If I wasn't building NodeJitsu, that's a big one because I've been doing that for more than three years. I would probably still be writing Node software somewhere. I would probably still be working on open source things. That's what drew me to Node and to start Notizio in the first place. And I guess, Isaac, if you weren't in the middle of this crowd fundraiser and doing what you're doing with Node, I think you answered this a couple shows back, but has your answer changed? What else would you be doing?

What was my answer last time? I don't remember even. I think you said you were going to be sailing. That's been a popular answer. No, no, no, no. I wouldn't have said that, I'm sure. I'm terrified of being out in the ocean. It will kill you. It's full of monsters.

Make a new one up. What would you be doing? What would I be doing? I don't know. I would probably be going to yoga practice more and, I don't know. I would probably be going to yoga practice more and, I don't know, maybe living somewhere warmer. I think in 101 you were talking about how it was kind of happenstance that you didn't have a job and it was kind of like you had this extra two or three months just kind of sitting there and you're like, ah, I'll build something. So I guess maybe it would be around that. If you'd never actually built NPM and never got into Node and took over the maintainership of it.

Yeah, I don't know how long I would have gone without a job. Maybe a year or so. And then I would have run out of savings. And I don't know. Probably gone back to Yahoo or something. Back to Yahoo. I mean, we're pretty lucky as programmers, right? You can just go get a job whenever you want one.

Not a lot of people have that luxury. Yeah, that is absolutely true. I mean, yeah, that's totally true. The bad part about that is there's lots of jobs, not always lots of jobs you actually want to do. Sometimes there is, and maybe if you're you, you have, you know, better pick of the litter, but not everybody has that, that luxury. But how about Programmer Hero? Charlie, we'll let you go first. Who is your, who would be somebody that was very impactful to you

over the course of your career, Programmer Hero? People have asked me this before i don't have programming heroes um yeah maybe there's somebody that's been important on another show i do called founders talk i'd ask somebody you know who's your founder here who's been like a hero to you to help you get to where you are today um i like scientists i'm i'm big into that whole thing, the sort of mountain of work that needs to be done to inch society forward just a little bit. And in that sense, I don't know. That's a tough question. The big ones, Newton, Einstein, those sorts of things.

But people living heroes, I've never really identified with many. Newton was kind of a dick. Newton was kind of a dick. He was kind of a dick. He and Leibniz really went into it. Yeah, but I mean he had the – Oh, yeah, way before Leibniz. I mean he even – you know he wrote more about the Bible than he did about physics or math or anything. Yeah.

He was searching for Bible codes to try and like tell the future. See, I mean that's – these are the things I learn when I hang out with Isaac. There you go. And yeah, I don't know. Isaac, what about you? I don't know. I don't think, in our show notes, we didn't have one for you last time,

so I'm not sure if you didn't answer it or we didn't ask you, but there's nothing in the show notes. My programmer hero today because um i i just recently had to patch his code and and i i really liked the experience is uh trent mick who is uh he's an employee here at joint he wrote dash dash which is my new favorite uh options parser for the command line i'll have to check that out yeah dash dash is super neat cool is it spelt out or is it literally

like like underscore type thing dude come on like you can't publish hyphen hyphen as an npm package name you can't i don't know i thought maybe you could no way it's got to start with a letter or number uh but uh no it's spelled out d-A-S-H got it and is that Trent Mick or Nick Trent Mick with a M as in Movember gotcha and it is Movember isn't it are you guys Movembering it

no I don't like mustaches I think that you should just give money to prostate cancer research if that's what you want to do. Don't grow a mustache. It's disgusting. I'll pay you $10 to not grow a mustache and you can give that to prostate cancer research.

Oh, boy. Wow. No, I'm just kidding. I won't do that. Now everybody knows how to get an extra $10. Yeah, there's a lot of people out there. And a lot of them have mustaches. But no, I am not.

I shave my face like a gentleman. Gotcha. Cool. Well, guys, I want to thank you for joining us today on the show. Definitely – we as The Change Law definitely want to support you however we can. It's not quite the future yet, but tomorrow morning we have a post planned to help. Obviously we'll post this podcast everyone's listening to as well, but we want to support you however we can. And we think that you should too, and you can go to scalempm.org to show your support. They said don't do it, but I say put your money where you put your packages.

I mean don't put your actual Bitcoin in there, but definitely help support this for sure. And for all of you companies, corporations out there that are using Node that are listening to this or someone who works there, share the information upline to get corporate sponsorship in there and make this thing happen. So Isaac and Charlie, I definitely want to thank you for coming on the show today. I want to also shout out to our sponsors, DigitalOcean and TopTow. Something cool today that DigitalOcean just mentioned was a one-click application for Doku. We had Jeff, Lindsey on the show a couple back. If you haven't listened to that, I'll put that in the show notes. But super cool. There's a one-click install application.

You can like boom in one second have a droplet with Doku on it already. And you can take advantage of our $10 hosting credit. Use the coupon code changelogsentme. That's changelogsentme. To use that, you'll get a $10 hosting credit. And if you like to write tutorials and you – I just saw Jeff actually released a community tutorial that went along with this. But if you're like Jeff and you want to write a tutorial for DigitalOcean, you can get paid $50 to do that.

We'll have links in the show notes for that as well. And it doesn't matter where you live. If you want to email Barry at DigitalOcean.com, he will send you stickers, DigitalOcean stickers. So take advantage of that. But Isaac, you mentioned earlier about having the. So take advantage of that. But, uh, you know, Isaac, you mentioned earlier about having the opportunity to work pretty much anywhere and, and our partner and, and, uh, sponsor TopTal, uh, is, is, uh, able to make that happen as well. You can join their team and a network of awesome people from anywhere in the world and, uh, and work anywhere

basically, uh, with TopTal. We, we mentioned how they do some pretty cool freelancing, but you can go to toptile.com slash developer to apply. And if you haven't yet, check out their TopTile engineering blog, which has been featured on the show before as well as in our newsletter, but toptile.com slash blog for that. But, guys, anything else you want to say before we make this a wrap no i'm i'm really just looking forward to see uh how this thing uh plays out you know we've been it's been a sort of whirlwind uh since uh couchconf when this whole thing happened and it's uh exciting to see it

wrap up this way yeah it's i like when you kind of get a problem and you kind of figure out how to solve it and then you release it then it's solved and it's like wow yay we did it yes it's that moment or at least tomorrow's the beginning of that moment for you guys right so we're getting that's the idea that is the idea well fellas thanks for uh joining us on the show today we certainly support you however we can uh let's say goodbye take Take care. Thanks for having us. you