Programming Throwdown - Open Source Communities

Episode Date: May 2, 2016

On this episode we invite Mikeal Rogers, Community Manager at the Node.JS Foundation, to talk about open source communities and organizing large open source projects. ★ Support this podcast on Patreon ★

programming throwdown episode 53 open source communities take it away jason hey everybody so today uh we have a really cool interview. We have Michael Rogers from the Node.js Foundation. And Michael, why don't you start by introducing yourself? Hey, I'm Michael Rogers. I do Node stuff. That's me. So what's your role in the foundation?

So my role is, my title is Community manager, but I do quite a bit. So part of it is community work still and working with, we have a new governance structure and a TSC and a really big flourishing project that I'm sure that I'll get more into in a little bit. So I work with that on the community side and just kind of try to make sure that any roadblocks are pushed out of the way and that, you know, people are being connected when they need to get connected. And then I also actually do a lot of the internal foundation stuff. So we have a lot of resources from the Linux Foundation around marketing and PR and events that we use. So coordinating a lot of that, putting together, you know, board meetings and working with the board of directors and all that kind of stuff too.

Cool, cool. That sounds almost like a real company. I say that sarcastically. Well, it's yes and no. I mean, I think at a real company, you might have a boss. And everybody's my boss, it turns out. The board of directors are my boss.

The whole community is my boss, and then I also have a boss at the Linux Foundation. So yeah. Cool. I mean, that's the hierarchy, right? Everyone's each other's boss. Right, right. That's one way to look at it. So what is sort of the mission of the Node.js foundation? Like what is sort of the goal of the foundation? Well, the foundation sort of the goal of the foundation? Well, the foundation is there to basically make sure that Node is stable and that it, you know, meets its potential.

You know, it actually reaches its potential. You know, Node has the possibility to be, you know, the largest programming language in the world and do a lot to change computing. And to make sure that it actually, you know, facilitate, like, that actually happens, we have to facilitate, you know, a healthy project with a lot of contributors, with a lot of stability and security considerations. We have to, you know, support a growing ecosystem. And we also have to, you know, support kind of more traditional marketing, PR, analyst relationships the same way that proprietary alternatives do, right?

So if we're really going to change computing, we're actually going to need to compute with some of the proprietary technologies out there. Cool. So, oh, real quick. So we covered Node.js in episode 41. So if you're here to, you know, learn how to start your own server out of Node.js or something like that, go to that episode first and then come back to this one

and learn about sort of the community that helps make that possible. Yeah, that's a good point. I don't know how much we'll get into Node.js technical details as opposed to using this resource this time with Michael to learn about open source projects. A lot of people, and we recommend people get involved with open source projects. But I'm curious, like, you curious, I start something on GitHub. It's Patrick's best new compiler for some new language, Patrick. See, I'm very ritual in my name. And it's just me working. Obviously,

that's not a foundation. There's no structure around that. What's the path of growth from the language Patrick starts getting adoption? And then like, could you kind of take us maybe through about a little bit of how something grows from me working in my basement and like by myself to a whole organization? Sure, sure. I mean, I think that, you know, most projects that we put out there don't go anywhere, you know, like, you know, even, even if you have a bunch of Twitter followers and get a followers, you know, most of the code that you put out there don't go anywhere. Even if you have a bunch of Twitter followers and GitHub followers, most of the code that you put out, nobody else uses. And part of modern open source is just kind of publishing everything and seeing where it goes.

So I have a lot of modules that are really heavily dependent on that a lot of people use, but I also have hundreds of modules that nobody cares about. And so putting stuff out the door and just engaging in the open source community is kind of the first step, right? So you just get it out there, you see what happens. If you start to get users,

if people actually start to depend on this thing, you're going to end up getting contributions back, right? And by contribution, I really mean any kind of engagement from another person. So they could comment on an issue, they could, you know, even people that ask for help are engaging with the project in some way. And they may send you, you know, little fixes,

big fixes, documentation fixes, that kind of stuff. And it's really important that as your project grows in users and as you start to see some of these new participants coming in, that you actually turn those people, at least some of them, into longer-term contributors to the project? People that may actually help review different contributions that come in or potentially in the future help you kind of run the project or do builds and things like that.

And this is the thing that I think most open source struggles with right now is that when projects start to grow, we get stuck in this single maintainer phase where it's just one person trying to handle all of the onslaught of contributions that are coming in. And that creates a lot of tension with your community because you have people that really just want help and you're just getting frustrated by the amount of help that they're trying to bring. And so we're really trying to develop a culture and a skill set and a set of guidelines for developing contributors that come in

into longer-term assets, at least some percentage of them, and also making sure that projects are in a state where people can easily contribute to them and it's not very hard, so you get a lot of these smaller contributions coming in. Yeah, that makes sense. I think one thing you mentioned

that kind of really resonated in the beginning is that, and this is true for anyone, if you do work in open source or you publish anything, most of it is going to be kind of thrown away. And I think,

actually, someone, Ira Glass, did you see his talk on the creative process? Yeah, yeah. And there's some really choice quotes in it, too, just about how it's really hard for people when they first start putting out creative stuff as well. Because you're putting it out because you're kind of a fan first, and then you decide to make something. And the first stuff that you put out is just terrible. It's not up to your standards as a consumer. And you have to get over that and just keep making things and keep publishing things.

Yeah, definitely. So going back to Patrick's, I mean, if you love doing a certain – like how – at what point do the people making Node.js say, okay, now we need a foundation? Like actually like where does the – I mean at some point companies have to be involved I suppose like to get funding and things like that? Well, I think it really depends on the needs of the project. And every project's a little different. I think that jumping to money is probably a mistake for most projects. Money and adding kind of new incentive structures can do a lot of harm to a project, to be honest. Like, for instance, if your project is really hard to contribute to, and you decide to go out

and raise money to pay people to work on it, you're not actually going to end up making the project and the easier to contribute to. You're not going to end up getting a bunch of new contributors. It's going to retain all of those barriers to entry that it always had before. And in fact, a lot of potential contributors may look at that and go,

well, you know what, this is maintained by people that are paid. I'm not going to even, you know, spend my time trying to engage in it. So there's a lot of kind of counter incentives that can happen too. Yeah, yeah.

That's kind of what happened with, I believe, with Open Office because Open Office was just so heavily run by one or two industry. I think it was IBM and Sun, I think. And then no one else. There's a very strong barrier to contribute. Yeah, yeah. no one else like there wasn't there's a very strong barrier to contribute yeah yeah and and i mean you know apache has some rules about trying to uh forcibly make the contributorship diverse but there's not a lot that you can um you know really say it at the kind of process level about keeping it easy to contribute in the node project we've done a lot

to make it easy to contribute um and to reduce all these barriers to entry, right? So, I mean, for one thing, the project is split into a lot of different kinds of repositories so that if you have a skill set for working on websites, you can contribute to the Node project. You don't have to be a core low-level C hacker to get involved in the project and to really make a difference. And we see a lot of people kind of level up through that process, too. They'll work on, in the evangelism repo, making blog posts, and then that'll get them into the website. And then from there, they'll go, oh, well, what does core look like? And they'll do some documentation or some test fixes.

And then all of a sudden, they're sticking around and really getting deep into the code. And we have a record number of contributors to Node now. We have, I think, over 70 committers. So about a year ago, it was five. Or like a year and a half ago now, I guess. Just the TSC, the governing body around core, I think, is like 18 people now. There's just a ton of contributors. I think in the latest release, we updated the author's file from about three months ago,

and I think like 120 people were added to it um that are just outside contributors yeah and a lot of it is just you know making it really easy to contribute i think that you know if you look at um the github data on this like the majority of work happening pushes and commits that are going into the github ecosystem are coming from people that commit like less than 20 times a month, right? People that are not super highly engaged. The people that are highly engaged account for like a shrinking percentage of the total workload. And I think that this is really hard for projects to kind of wrap their head around because the people that show up every day are the people that have a voice and that tend to have power and that tend to create the process in most projects.

And it's very hard for those people to create a process that's not for them. It's actually for people that are less engaged because the people that show up every day are going to show up every day no matter what. Oh, I see. So you've got to actually make it easier on other people. And obviously, I mean, you know,

I don't really think that there's a one size fits all process for this. You know, if you look across the Node.js project, we have different policies around contribution in different repositories because, you know, the Node core project just has different constraints than the, you know, website. Right. And so we can make the website incredibly easy to contribute to and not, you know, worry about things like, you know, comment metadata. But in the core project, we really need to make sure that that stuff is really clean because we have to do all these branch merges and all this other complicated stuff. Oh, I see.

So what if you have to do something kind of atomic, like I'm trying to think of a good example, like HTTP three comes out and, and you have to update to HTTP three. It's a massive change and kind of everyone has to be on board because it changes everything. How do you sort of coordinate that when you have so many people all across the globe?

Right, right. So yeah, that, that touches on a lot of things that we have in, in the node core project. So I'll touch on a couple of them. One is that when a new contribution comes in from anybody, the default assumption is that it will land. So we don't say, you know, convince us that this should make it in. It's actually up to all of the committers. And now we have over 70 committers, right? It's up to them to say reasons why something should not go in and basically have corrections. So the review process is the series of corrections

to make this thing land. And there's sort of like a benefit of the doubt thing, right? We also bend towards things happening. So we don't escalate, we don't ask for votes, anything where somebody's not objecting to it, Right. So until somebody says, you know, this probably shouldn't go in or I have a problem with this. That's when things get escalated and we actually bring them to a vote in the CTC, which is basically a bunch of committers that are a little bit more trusted with the technical parts of the project. So this is like called like a consensus seeking where um you try to reach a consensus if nobody objects things just land um and that's over 90 of the contributions that come to court just land that way they don't have to get escalated they don't need to vote uh we don't have a lot of like burdensome process in the way of getting things in um also part of that whole review process is

that um changes need to sit for like between two and three days, depending on if it's a weekend or not. And that's because we have people all over the world in every time zone that need to have a chance to review code that's coming in. Oh, that's true. I didn't think about that. But yeah, if you get someone in Germany who really objects and you send it off at, you know, 3 PM Pacific time, that person's voice won't be heard. Totally. And this at 3 p.m. Pacific time, that person's voice won't be heard.

Totally. And this is really important for us. I mean, we have a lot of skill sets around the NodeCore project. I mean, most people don't have a crypto expert, and we're lucky enough to have two people that really know the OpenSSL code base really well. And they're in Moscow and Japan. So we need to make sure that, especially when security stuff is going in,

that they have a chance to weigh in. And, you know, I mean, the TSC lead is in Australia. One of the longest contributors is in, like outside of Amsterdam. You know, obviously there's a bunch of people from the US and other countries in. So we just have people all over the world, and we need to make sure that everybody can participate

in the reviews. So that's been really, really critical. And I think one of the things, too, is that if you look at the Node project, we don't have a really stringent roadmap of, like, this feature is going to land on this date, and then this feature is going to land on that date um our roadmap just says you know this is when we're doing our next uh major release that will probably be a breaking release um and this is how long our long-term

support cycle is um and this is actually this allows anybody to get anything in because we're not saying you know this stuff is more important than stuff it's if you care about something you want to get it in, it can probably land. If you want to get something into Node, send us a pull request, get things moving, and we'll work on getting that code in. And this has led actually just to a lot more people getting stuff in and a lot more even really big changes just because people know that they're not going to be pushed in the back burner just because it's not part of some roadmap. So that works for the bottom-up kind of changes. But yeah, like Jason was asking about a more overarching direction. How do those things get set and how does that get coordinated?

Right. a lot of that because we tend to take an ecosystem approach to the project where we want a diverse and flourishing ecosystem on top of Node. We don't want Node to become everything to everyone. And those are always kind of count like, you know, those are conflicting forces, right? Like, I think that one of the reasons why Python doesn't have a huge module community is because Python core continues to grow this giant standard library. And one of the reasons why we have the largest module ecosystem in the world is because we've kept core very small. So we don't do a ton of giant shifts in one direction or another.

We do have certain large-scale, high-impact features. One would be this feature called AsyncWrap, which is about really low-level instrumentation and debuggability that could completely destroy performance if it were done wrong. We also are talking about how we might add support for ES6 modules, and that's a big discussion. And for those things where we really need to get alignment about this before we sit down to write code, the way that they're done is with this thing called an EPS and an improvement proposal, or enhancement proposal, sorry. And it's just a spec. It's a markdown spec where we, and we do it as a pull request in a separate repository. And then once the pull request kind of, we use the same kind of process that you would for reviewing code, it's just the spec instead.

And then, you know, that eventually goes up for the CTC, so that, you know, the project can say yes is the direction we go, yes we want to add this before anybody goes and sits down to write hundreds of thousands of lines of code. Okay, that makes sense. What about, like,

do you guys ever meet face-to-face? Like, do you have a convention somewhere and say, hey, everyone, all the Node.js committers can come to this place at this time and hang out, grab a beer or something? Well, we have a few too many people to claim that all will be at anything. Right, right. It's just a little bit too big. But yeah, so the foundation runs two major events this year,

Node.js Interactive in Amsterdam and in Austin. And part of that is a collaborator summit, which is more of like an unconference where people who are active and committing and working on the project just get together and try to hash some stuff out. And we're also doing this thing called the Code and Learn, which is that if you want to work on Node Core

and you want to get deeply embedded in something, but you want just like a day's worth of mentorship to kind of dive in, we have all the contributors there that can help you do that. And then, of course, we have all the regular kind regular talks by core people and stuff like that at the regular conference that happens as well oh very cool so we get one of our most common

questions, email questions in are people who want to work on an open source project but they've literally just picked a book on C++ 101 and so then they try and jump into Node.js for example and they don't really know where to start.

Is there, I mean, what would be your advice to somebody who maybe just graduated from college, knows nothing about Node, but they want to sort of donate their time? Or maybe they're a Node user, but they use Express or something, but they know nothing about the Node code base, but they want to help. What's something, even maybe something small that they can do to get started? I tend to look at this in terms of barriers to entry.

There's two categories of barriers to entry. One is technical and one is cultural. So if you want to work on a new project, you may have to learn technical things in order to get over a hump. But then you also have to acclimate to how this project works and how things happen. And it's a little daunting to go after both of those at the same time. So I tend to encourage people to look at the project and look at their skill set and do something that they already know how to do, right? So you can adjust documentation, you can adjust a test pretty easily, basically anywhere. You can work on the website, you can help write a blog post. There are all these different areas in the core project

that you can contribute to where most people that will listen to this podcast will have some kind of skill set around web development or documentation that could do that. And, you know, that's like a good way to dip your toe in the water. And that's a good way to only acclimate to the cultural part and really just kind of figure out how things work. And, you know, make sure that this is like a nice welcoming community that's not, that you actually want to be a part of. So I would encourage people to do that first. And we have a couple tags across the project, like, you know, good first contribution and things like that, that are, you know, good kind of low hanging fruit. But yeah, that's what I encourage people to do for sure.

So if you want to acquaint yourself with the community, with the cadence, with the direction, with that kind of stuff, is it typically that open source projects are using mailing lists for that? IRC chats? You want to be an observer. I want to get interested, but like you said, before I invest all this time,

I want to kind of get up to speed because there's this embarrassment thing that you're worried about, right? Like, I'm going to send an email and this person is going to think I'm really stupid. Now, there's various issues with that, but, you know, that's kind of how people feel. And so you want to kind of sit on the sidelines and just watch what's going by before kind of stepping in and helping. And what is the best way to kind of figure out where the quote unquote community lives? Right. So for us, everything is on GitHub.

We actually, we don't have mailing lists we we basically treat the github issue tracker almost like a mailing list um and uh inside of the node js github org which is just github slash node js there's like a hundred repositories right and um you know each one kind of covers a different area um the best way to figure out where your thing is that you might want to be interested in is that there's a repository just called help. So just GitHub slash Node.js slash help. And in there, you can ask any question that you want. So it could be about, I don't know how to do this thing in Node.

Or it could be, you know what, like, I have this kind of skill set. What are the things that I should, like the repositories that I should watch? Or how should I kind of get involved in this particular area? And people there will tell you how to get involved. So the answer to that question really depends on the individual person's skill set and what they want to work on. But we do have a place where you can figure that out. Nice.

So you brought up something else, which is I'm using Node.js and I have a question. And what does kind of the Node.js foundation of the project do? Because obviously for it to continue to be important, people need to be using it. Like just developing Node.js is awesome, but if no one uses it, kind of what's the point?

Like, I mean, there is a point, but like past a certain level, right? And so how do you kind of help try to make sure that the people using it are facilitated, have the help they need, have a healthy blog post about how do you write projects in Node.js, as well as contributing to Node.js? Right. So I think this is actually one of Node's strengths.

And I mean, we have 100% year-over-year growth right now. It's tremendous how fast we're growing in terms of users. And a big part of that is that we have a wealth of almost anarchistic resources about learning Node and doing things in Node. And that actually turns out to be a big asset because different types of developers just look for different types of information

when they're working through things. There are certain developers that will just bang at something until it doesn't work and then they'll Google something. And then hopefully that lands them on like a stack overflow article or a blog post that's like related to what they're doing um and we have a ton of those kinds of resources up we have um you know we have api documentation we have node school which is like this tremendous community of people putting together uh self-guided workshops and also just around the world

doing little meetup workshops and getting people into Node and learning Node. So we have all of these different types of resources for different types of learners that people tend to find. And if you're looking for, if you don't know what type of resource to find, I would go to that Node.js slash help repo and just ask there.

And you'll definitely get a good answer. But yeah, I think that, you know, a lot of projects tend to focus on, you know, one true answer for this kind of stuff. Like, you know, we have perfect documentation, and it's all right here. And, you know, we have API documentation. But we also have, you know, just a million things that you find on Google, from traditional published books to, you know, self-guided workshoppers to just stack up with little articles. Nice. Yeah. And then what about establishing like standards? You talked about Python. So Python kind of has their like, here's the way to write Python. Now you can write

it however you want, but you know, they kind of have, right. Yeah. So they kind of have, I don't know exactly their relationship of pet bait to them, but there exists a standard. Let's pretend they did it for now so I don't have to Google it. But that's kind of one attitude. One attitude is to say, look, I don't care how you write it. Do what you want. But obviously people who know what they're doing are a good resource to say, what's the right way to structure a program, to have module decomposition, that kind of stuff.

How do you help shepherd the community from that standpoint? So I think Node stays out of this more than probably any other language. There's a lot of creativity in the JavaScript community, and we don't want to tell people what our opinion on semicolons is and try to say that that's the one true way or anything like that. I think that there are, though, a few core patterns that Node defines, and a lot of compatibility with Node itself and with modules in the ecosystem

depend on those patterns. So if you're doing iterative processing of data, you would use streams, and that's really important. There's sort of a standard callback API that all the Node APIs use and that most modules in the ecosystem use, and that's a really important kind of core pattern so that people can use that. That's just the error first callback pattern.

It's very, very simple. And I think, you know, for publishing modules to NPM, there's something of a best practice there, which is that a module should be a single function and should, you know, do something very well. And so most modules that you see published to NPM are a single function that do a single thing. And they're usually composed of like 100 other modules to get that thing done. But those are really the only patterns

that we kind of push and advocate in the community. We don't push a particular style guide. We don't really even have an opinion about higher level constructs like promises. We're really kind of trying to stay out of that and let the ecosystem decide what they want to do. The only other thing that we do a little bit more proactively

is that as new language features land in V8 and then we take the versions of V8, we do a fair amount of work to figure out what is optimized and what isn't and what is fast and what isn't. Mostly just so that in Node itself, we can know which features to use and not to use. And then we let the community kind of know which of those

features are already optimized and which are not. So that's actually interesting. So you talked about V8. Maybe could you talk briefly about kind of Node's relationship to V8? And then also like V8 isn't Node, you know, at least in my understanding. And so like depending on another project, right,

has its own good parts and bad parts, and talk about how Node manages that. So I think V8 is the JavaScript engine that Google made for Chrome. And Node has always bounded this VM and has used it for a very, very long time. Every major release of Node usually comes with a new V8. And that, up until this last one,

that meant that the C++ API would break. The Chrome team, like the, because the Google manages Chrome and V8 kind of together, they don't mind breaking their binding layer because they can just update the Chrome code. But in the Node ecosystem, if you have

any native modules, anything that's written in C, C++, this causes a big problem, and it takes a while for the ecosystem to upgrade. So that's one of the reasons why Node has a longer major release cycle than V8 does. V8 has a major release every six weeks, and we only have one every six months.

But for a while, there was really not a lot of communication between V8 and Node, to be honest. It really kind of broke down a couple of years ago where it got pretty close to zero communication between the V8 team and Node. But really, since IOJS and since IOJS merged into the foundation and there's really a foundation in place,

the V8 team is getting more and more integrated into the Node process. And in fact, certain resources from the Google Cloud platform, which is heavily investing in Node, are also working on Node for Google that aren't specific to the VA team. So that's really created a better relationship with us. And I think that they care a lot more

about that kind of API stability. And we're working with them to maybe bring an end to these breaking changes to the C++ API. So that's actually going really, really well. And in addition to that, at the same time is that you have the Chakra team from Microsoft binding Node to Chakra, getting Node onto the Xbox and to Windows 10 IoT by binding it to Chakra. And you also have the Spider Node project, which actually started, I think, back in 2010, but is now actually updated and working again for the first time in like five years.

And that's binding Node to SpiderMonkey. And so I think in the future, we'll probably see a slightly more VM-neutral Node.js than we have in the past, although I don't see any time soon where the default won't be V8. Interesting. I've done a lot of Node projects. I'm a huge fan. And I saw IO.js kind of come and go on Hacker News.

I never actually took the time to find out what it is. It's some kind of fork of Node.js, but now it's merged back. Can you kind of talk me through that? Sure. So there was a disagreement with the owner-steward of Node.js, Joyent, and some of the contributors

that were in the community contributing to Node. And the disagreement wasn't about technical direction, really. There were clearly a lot of problems that Node needed to solve. But the real disagreement was just about how the project was run, right? Is this a governance structure? Is this a BDFL model? How do we have

more of a voice in this thing that we're dedicating a lot of our time to? For people who don't know out there, so BDFL is Benevolent Dictator for Life. And so that came from Python where Guido had the final say on everything. No, it came from Linux. Oh, it came from Linux. Yeah, yeah. Linux is the BDFL of Linux.

Oh, okay. Yeah, yeah. But yeah, and so Node had a BDFL, but it would turn over every two or three years. So it was Benevolent Dictator for two or three years. So it was a benevolent dictator for two or three years. That acronym is way harder. Right, right. And I think as the community around Node grew, the user base grew,

the amount of outside contributions kind of coming in grew, the number of committers and maintainers was not growing. It was not keeping up. So the model of managing a project was just not scaling with what was happening to Node. And so the people behind IOJS thought that they had a lot of good ideas about scaling out the project,

really learning from Node's own ecosystem. There were a lot of great kind of governance experiments in the Node.js ecosystem. And so they created this fork under this other governance model. Most of the committers to Node went over to IO.js. And that model worked incredibly well. I mean, it, you know, I think in three or four months, it became, sorry, it hit a record high in the number of people contributing to Node.js. So back, like, you know, it was like 2012 since it had been that high. And then they eventually kind of broke that record. So it was a tremendous success in terms of bringing in new people to the project.

You know, it wasn't super stable, right? Because, you know, doing really fast iteration, taking V8 all the time, doing all these major breaks. But because it's not Node.js, and, you know, people were kind of opting into the fork, didn't really have to deal with the long-term stability issues that Node had to deal with because it didn't have any history yet. But at the end of the day, there was no technical disagreement. There was no like, oh, we want Node to have more features or we want Node to go in this direction or that. It was just about how to manage the project. And so Joyent decided to start the Node Foundation and put Node into

a foundation. Once Node was into a foundation, the IOJS group decided to merge in. And essentially, the foundation for Node.js adopted more or less the governance model from IOJS. So all those contributors came in, all that process came in. But now a lot of new things needed to happen in terms of release scheduling and long-term support to bring all of these new contributors in this community to support something that needs to remain stable for years and years. Right. Um, so, and it would, I mean, it's been a huge success. Like it took a lot of work from, from,

you know, me and James Snellen and Rod Vegg to make all this stuff happen. But at the end of the day, we have, um, both like the, the stability of Node.js with the competencies of open governance and a huge amount of a growing contribution base from IO.js. So we really got to take both of the core competencies and merge them together. So that was great. Cool.

Cool. Totally makes sense. So IO.js now is merged in. So you get all of those features by using the latest Node.js? Yeah, so as of version 4, the last IOJS version was version 3, and so version 4 of Node.js was the IOJS merge, essentially. Gotcha. Cool. Very cool. So what is, you know, you're a community manager. And so someone who's community manager of an open source foundation, what's sort of your day to day like? What are sort of the things that you kind of typically have on your plate? Or just in general, like the Node.js foundation? What are kind of the day-to-day activities there? So, I mean, with a community that's growing, you know, it's doubling every year, that kind of always changes, right? I think, you know, really early on, I was much more involved in the kind of day-to-day of the project because we were doing this huge merge to bring these communities together and we were still kind of

coming up with the practices and processes by which we could continue to scale the project to more and more and more contributors. Once a lot of that was in place and there were like a lot of like kind of new leaders established, I've been able to pull away from that quite a bit more. And so a lot of what I've been doing lately is working with the kind of team internally and externally to come up with a really consistent message and understanding of what Node is and why it's really successful. I think that the traditional view of Node.js is that it's this thing to write servers in. But if you look at our actual usage and our actual growth, we're so much more than that. I mean, the way that people develop front-end applications is now totally different, right?

Like, nobody used compiler chains unless you worked on Dojo, like, seven years ago. And now everybody doing modern open-source development is using a compile chain written in Node.js. I mean, now, like, the new front-end framework is not a thing that you stick in a script tag. It's a thing that you use in a compile chain written in Node.js, like React, right?

And so, right. So front-end development has totally changed. You also have Electron being used by a huge amount of desktop applications now. I cannot believe that a desktop framework took off this quickly. And is now, you know, like Node is now the easiest way to build cross-platform desktop applications. Yeah. For people who don't know, we covered Electron in our tool

of the show. But actually, we covered native of fire in our tool of the show, which uses Electron. And that was a tool where you could just say, you know, google.com, and it makes a, like an app, like a OS 10 app for Google, like you could have the Google app, and it has an icon here, just some things like that. And so electron is a way to wrap a website into an application. And I didn't even know that was written in Node.js. But it makes sense when you think about it. Yep, it's I mean, it's not in WebKit. But I think a lot of the power there is coming from you know using web technologies to create the whole front-end experience but then having node um to handle networking and file system and a lot of other stuff um so i mean a lot of people don't

realize but the slack desktop application is written in electron the um microsoft's visual studio code editor is written in Electron using Node.js. Yeah, yeah. It feels native, and so people aren't really recognizing when this kind of stuff happens. Kind of the way that a lot of apps that you use on your phone you think of as iOS apps are actually written in Cordova using web technologies, also using Node.js actually now as well. Yeah, yeah. And that that's a big part of node um cordova is a big part of node doing you know mobile and tablet applications um also

iot is really taking off so um last year all of the new iot embedded platforms now run node on device right so even low power um so that's just been like a tremendous shift. And then also, all the new cloud technologies, the serverless technologies like Lambda and Google Cloud Functions and Azure Functions, they're all on Node as well. And sometimes Node is the only platform that they're supporting right now. Yeah, yeah. And then, you know, Node is huge in enterprise now as well. The microservice stacks, 90% of what people are running on top of all that stack is just Node.

So when we started looking and really breaking down why is Node so successful in all these areas, where is the growth coming from, it's really kind of self-reinforcing. We talk a lot about Polyglot and a lot about the best tool for the job. And I think that if you're a really experienced developer, that's an amazing luxury to just be able to run anything

and be able to select a language. But the vast majority of developers just want to get a thing done. Like learning a language is another thing in the way of them getting something done. And Node being this much more transferable, universal skill set across IoT and back-end and front-end, it really kind of lowers the barrier to entry to building these bigger, complex applications and this really, really big new full stack that you have to contend with.

And having a unified debugging environment and a unified tool chain for when you're working on an IoT device or on a front-end application is really huge. That makes sense. So a couple of things. One is, these are kind of more kind of nits, I guess, about Node that I'm curious. One is, I know ES6 has its own type of, I guess, require. And so will Node kind of adopt that? Because one kind of issue that I think Browserify was able to do pretty well, there's also RequireJS and some of these other ones, was that when you wrote something in Node,

that code didn't just also work on the browser. Like if you had something like a combinatorics library where you had to do it in Node, but then you also needed something on the browser. But you had to do all this crazy stuff to wrap the code so that it could run on both. And are you sort of working with the browser JavaScript community to try to make cross-client server development easier? Well, I think, you know, the ecosystem is really figuring this out for us. I mean, I think that, you know, when Node started,

a lot of the problem was that the majority of JavaScript that anybody had ever seen was written for the browser and relied on some DOM APIs that we didn't have in Node, right? So Node was like this new kind of virgin platform and didn't have any modules on it. Now Node has more reusable modules than anywhere else, right? And so there's this huge amount of incentive for people to come up with tool chains

that make all that work in the browser in a variety of contexts. As far as ES6 modules go, that's sort of like a much more complicated question than I wish it was. I think that today, a lot of people are already using ES6 modules just through a compiler chain, especially if you're doing front-end stuff.

But even some people doing back-end stuff are just using a compiler chain because they like ES6 modules. And there's not a huge amount of value that gets added to that experience by having it work natively. And I think also a lot of people are still going to rely on those compiler chains if they've opted into them today, because there's going to be new features that still aren't in V8 yet that are like the next feature that people want to play with. Right. Um, I think a lot of people that, you know, want to see a lot of this stuff come in natively, they're also using a weight, so they're not going to dump that compiler chain anytime soon.

Right. Or they're using async await and other features that aren't in V8 yet. And we don't even know when they will land in V8. Can you explain to the audience out there, like, what's a compiler chain mean in this context? Right, a compiler chain is just something that takes your code and then turns it into other code that will run and execute in different

environments, right? So if you wanted to run in the browser environment, you would run it through the compiler chain that makes it work, you know, with the browser and not relying on a bunch of node APIs and adding, you know, intermediate modules or polyfills or whatever needs to happen to make those work. Gotcha. Right, right.

And yeah, yeah. And so there is a discussion right now about how to get ES6 modules into Node. And it's a much more complicated discussion than anybody had ever thought because there were some assumptions made about detection of what module type that you're in. That's handled by the browser by a difference in the script tag that we don't have.

And so the way that we detect that and turn it on is up for a very contentious debate right now. So I unfortunately don't have a great answer. Gotcha. So one other sort of... Before we start, keep spiraling down this, I want to go back to something you were saying before, which was,

so, you know, people start a new project, there's this right tool for the right job, and, you know, that's part of what the podcast kind of sets out to do is talk about

all these different tools so people know all the options out there. But in most big companies, you don't really get that luxury. I mean, it would be nice, but, you know, you kind of are limited to

C++, Java, maybe Python. If you're doing web stuff, there's JavaScript and stuff. But as somebody who is starting out or trying to transition into programming, Node.js has enabled people to, as you said, do more than just front-end website scripting

in JavaScript. Do you think that Node.js, JavaScript combined, has the ability to kind of be one of those core things that if you learn this as a first language, it has the legs to take you far into your career? Do you think it's there already? It will be there? What are your views on that aspect of Node?

I do think that it's there today. I mean, especially over the last year with the changes that have happened in IoT. I mean, Node is now supported on device probably more than any other language on IoT platforms. And that was kind of the last space that we were worried about being a fit for, right? It's hard to run on embedded hardware,

but it's happened. And a lot of the work that we've done being very, very efficient has really paid off there. I think, you know, to come back to some of the points that you made earlier, I think that I don't want to say that, you know, Node being a more universal language and people doing more kind of full stack stuff with Node is actually at odds with this notion of polyglot, because they're not entirely, right? So if you look at what's happening in enterprises, they're adopting more or less like these microservice stacks, right?

So they have Cloud Foundry, Kubernetes, Docker, whatever. But they basically have a way to build out infrastructure and then run anything on top of it. So they're not going to get vendor locked in the way that they did the last time that they did a big technology run 10 years ago, right? They want to be on top of it. So they're not going to get vendor locked in the way that they did the last time that they did a big technology run 10 years ago, right? Like they want to be on top of all these standards so that they can move providers and so that they can run anything. So the barriers to running a Go process in production, even though you've never run a Go process in production, is getting lower, even in enterprises, right? And that's a really good

thing because if somebody writes a really good analytics tool or a really good database in Go, they should be able to run that in their infrastructure without a huge barrier in the way. But the flip side of this is that deadlines are getting shorter. People have to do more with less. The communication overhead between having a team that only works on backend stuff and only works on frontend stuff is a really big barrier to getting things done. And so what we're seeing kind of across the industry, even in enterprises, is that Node allows people to reduce the number of people that are on these teams, have the back-end people and the front-end people working much more together. Even when you have a person that only works on front-end and only works on back-end, having them in the same team and speaking the same language is really, really useful.

And you have a lot more mobility and agency between people because they can do more inside of the organization and aren't limited by, you know, the language or the platform that they're used to. So tearing down these barriers is just like a really good thing. And one of the reasons why we're seeing such tremendous adoption is because we do such a great job of tearing down those barriers. I don't think that means that we'll stop seeing other languages and seeing people use other languages that are great at particular things. But when I look at what those companies might look like in the future, I really do think that Node is going to be the majority of what they run. But that last slice of what else they're doing is going to be a ton of different languages that

are amazing at very particular things. Is there a couple things you could give to say, if you're looking at Node to do this, you should look elsewhere? Since you just said it runs in most, I won't force you to say the things that mean you should use it, but let's ask the more interesting question of, hey, if my project involves,

like you said before, I spent a long portion of my life doing embedded hardware, so I would never have turned to JavaScript before. I do agree I see it coming out more. I don't know that I would still turn to it yet, but that's my personal opinion. But what are the things that are still out of bounds or not probably a good choice for implementing in Node? Okay, well, I do want to respond a little bit to what you said, though, about you using it, right? Like, one of the great things about Node running on these devices isn't that, you know,

it makes things a little bit different for people that were already able to write for those devices, is that it opens it up, like those devices up to so many more people that can now develop on it, because it's using Node.js and it's making things so much simpler, right? So it's really just reducing the barrier to entry to getting into those devices. In terms of what is Node bad at, I get this question all the time, and it's really hard to answer because in the early days of Node, the things that we said you shouldn't do with Node ended up being the largest growth areas for Node. So what do you want your next big growth area to be? you shouldn't do with Node ended up being the largest growth areas for Node. So what do you want your next big growth area to be?

There was actually a panel of all of the early Node core people at NodeConf, my first NodeConf actually that I ran in 2010, and sort of asked them, like, hey, what should you not do with Node? And they said, you know, anything that's like a bash script

and writing compiler and tools like that, don't use code right so that's one of the biggest growth areas that we have is in front end tools like that people used to do and things like bash and the other was databases uh and we actually have a huge amazing thriving uh bespoke database community uh built on top of the level db um those are both amazing areas of work for Node. So yeah, I think the one thing that if I look at other languages and I'm going to get jealous about, and every language does something amazing, right? They all have something they can learn from.

You haven't seen Whitespace. Yeah, you should check out our Whitespace language episode. Right, right. Most, you should check out our Whitespace language episode. Right, right. Most, I will say. If I'm looking around, though, and I actually get jealous of something, the fact that Go doesn't bind to LibC and that they went so far in the direction of everything just comes into this platform,

it's given them some really interesting portability that you have of the actual final compilation, which is really cool. And also, Libsyn's kind of garbage, so that's nice. And also, I think that they also have a custom crypto implementation, which is in Go. And this is one of the things that people don't get about, um, building out ecosystems and languages is that in the early days, people

will show up to build anything that you don't have. Um, and so it's actually, you know, like bootstrapping, like every, every JVM language does this thing, right. Where they go like, no, no, it's great. You can use all of the Java stuff, uh, and you don't have to switch over. And that's actually like not a great way to build an ecosystem because people don't show up to build like that thing in that language. And when Node started out, we were so incompatible with the whole world. So like you couldn't bind to any of the C libraries that everybody binds to because they all use blocking IO, right? So we had to write our own MySQL driver, our own Postgres driver. And we also weren't compatible

with any of the JavaScript on the web because we didn't have a DOM. That actually ended up being one of the secrets to our success was that so many people showed up and wrote these native implementations of everything. And Go has done that even a little bit more with their crypto stuff.

They also pay a guy at Google to write that. But yeah, I think that that's really cool. That's something really cool that they were able to do. And I don't think anytime in the future, we're going to have a pure JavaScript crypto implementation. So if I think about like, if I was thinking of like strengths and drawbacks, I I tell everyone the first thing you should learn is JavaScript you can build a cool website, you can do the backend in Node but I would say the biggest drawback for me for Node would be the weak typing

and we constantly get feedback about what is strong typing what is weak typing, we look this up and it turns out weak and strong typing is whatever you want it to be so but basically the fact that you just say a equals three now a is a string versus you know Scala with type inferencing Swift with type inferencing Java with strong typing right and so I know there's there's something from Microsoft, like a TypeScript, that might work with Node. But other than that, what are sort of ways to get good type inferencing or strong typing? Is Node looking into that?

So this is an area of deep thought. So a lot of the performance benefits that you get out of typing, we actually have in JavaScript, right? Like if you really look at how the VM optimizations work is that they figure out the types and they create like these inline, they do this inlining or they do some tracing or something.

So JavaScript VMs have a lot of ways of making things as fast as if you were using the typing. So there's not really like a big, in my mind, a huge performance thing to be said about this. I think that if you want... There are a bunch of people, though, that really like the early errors and the early type errors

and a lot of the changes that it makes to the way in which you program. I don't actually agree, me personally. Like, I mean, I started out with Assembly and C, and I come from a background of lower-level languages. And at no time when I went from, you know, statically typed languages to dynamic languages did I go, oh, man, I really wish I had type errors again.

That's just not, like, something that I've ever said. And actually, I do find that there's something of a correlation between people that get really into type safety tend to be people that started with dynamic languages and then looked at type languages. And it's kind of the opposite as you might expect. Oh, really? Yeah, yeah. It's really fun to watch. So yeah, I'm not personally a huge fan about that. I will say, though, I heard, I was talking to, I think, Tom Dale about this a while back, and he was looking at moving some of the Ember code base over to TypeScript. And one of the reasons that he cited, which I think is actually pretty smart, is that because all of the JavaScript optimizations are based on typing,

and if you end up switching the types too much, you end up getting out of optimizations. Moving a code base that you even expect people to just use in JavaScript over to TypeScript means that you're always going to use the same types and you're always going to hit the fast paths. And so it's a nice way to make sure that without sitting down and by hand going through all your code and figuring out where the JS VM is de-optimizing, you just get all that for free. And so that was a really good argument, I think, for TypeScript.

Oh, that makes sense. But you can totally use it with Node.js. I've never tried this, but I'm assuming you can go TypeScript to JavaScript and then do Node, your new JavaScript file, and you're good to go. Yeah, so the Visual Studio Code Editor is built in TypeScript. Oh, I see. Okay, and that runs on Node.

Yep. Very cool. Yeah. So we talk about how people can get started in open source projects, but also people new to programming are looking for a first project. You said there was some cool Internet of Things that will run Node. You talked about building, you talked about projects that can allow you to build Electron,

to build desktop applications. Could you kind of give some recommendations as people who want to get a taste of Node or are new to programming in general and want to say, what is there something cool I could build? Like some examples at various pieces of the stack maybe where Node is used and what people could do or look at there to kind of get inspiration?

So I think that if you're just sitting down to try to learn, the best resource is NodeSchool. So you can go to nodeschool.io. There's a bunch of workshops there that are like choose your own adventure workshops where you, you know, learn by doing and it tests your code and all that kind of stuff. And, you know, you can learn anything from, you know, learning basic JavaScript, basic node, uh, to Johnny five, IOT stuff, uh, to, um, there's even one for web GL. If you want to, you know, use the sack GL components that runs in the browser and does like 3d model diffing and stuff.

It's crazy. Um, yeah, yeah, no, it's, it's an incredible resource for, for learning stuff. Um, and there's also, um, a community on, on GitHub that will, you's an incredible resource for learning stuff. And there's also a community on GitHub that will help you get through anything. So if you get stuck, you can ask a question and people will help you get through it. So that's a tremendous resource that I point everybody to. In terms of, you know,

it really depends on what you want to build. I think that there are sub-communities around different parts of Node that you can get involved in. So if you're interested in robotics, there's an amazing community called NodeBots that does a lot of amazing robot stuff in IoT. There's a library called Johnny5 for doing IoT and hardware stuff that has a great community around it. It's actually built by one of the core contributors to jQuery, and so it has that kind of jQuery ease of use baked into it. So that's a really good resource. Obviously, the desktop application stuff, you should really look at Electron. I think the Cordova project is great if you're looking at mobile stuff.

Let me think here. Who else has a great community that I'm forgetting? Oh, if you're interested in WebGL stuff, there's a community called StackGL, which is small modules that implement all of the kind of fundamental math stuff around doing WebGL. They're great. And yeah, that's all that I can think of off the top of my head. Well, that was pretty impressive, actually.

I don't know. You should be ashamed that you could only name 30 projects. So I know you mentioned that uh obviously like money money doesn't solve open source community problems it creates more of them but but let's say uh my i start a company and my whole company runs on node.js and uh um how do you sort of uh like what what what can enterprise customers or users of Node.js do to sort of to help the community? I just go to one of these events is one of the ways. Yeah, yeah. For certain. I mean, that's that's also just a really good first step as well as to go to one of the interactive events. These are like the bigger kind of industry events as well. So they'll be very comfortable there.

I think it really depends on your level of investment and the level of dependence that you have as a business on Node, right? So, I mean, the Node.js Foundation has corporate members and we have an open door to any other corporate members that want to show up and get involved at any level. And that's, you know, how we fund the foundation and kind of keep the lights on. But it also gets you embedded into the sort of background marketing and messaging and

just the sustainability side of the project. So if you're heavily dependent on Node, that's a good place to invest. I think also, if you have the resources, and if you're highly dependent, having your people start working on node core, and potentially, you know, giving them more and more time, possibly even full time to be able to continue to work on core is, it's an investment that pays huge dividend, to be honest, a lot of, you know, a lot of the really big kind of node users have these people that, you know honest. A lot of the really big kind of Node users have these people that dedicate a lot of their time to NodeCore,

and they have an insight into how Node works and where things are going that have really helped those businesses. So, yeah, that's another really great way to get involved. And, you know, you can really get involved at every level. I mean, even the build infrastructure for building and testing Node. It's, you know, donated hardware and a bunch of open source contributors that manage, you know, this huge cluster and keep everything running. So, I mean, we're really, you know, we're there and have an open door for a lot of different types of skill sets. So you don't have to have a low-level C programmer to get involved in the Node project.

Well, I think we're approaching kind of the end here, but is there anything we haven't covered yet that you think people would be interested in hearing about? Anything you want to pitch? I think that we've covered pretty much all of our bases. I think that one of the things that we're doing as a foundation, and we're also working with the Linux Foundation on this, and I'm also putting in a lot of my kind of personal time, is trying to bring the public consciousness of open source up to where

open source actually is now. So we talked a little bit about how contributions are coming from people that are engaged a lot less, right? We have this, it's a, you know, not a long tail anymore. It's really like tail driven and the tail is the majority of the work. And the conversation around open source is always about kind of like these full-time open source people, a lot of ideas and argumentation about licensing and all of these things that really don't matter to modern open source developers. So we're really trying to push the consciousness of what an open source developer looks like just to the reality, not even trying to push it in any particular direction.

Yeah, that totally makes sense. I know that I've contributed to various projects, and one of them I contributed to recently was Gradle. And they made me sign this whole thing and fax it to them. Like, I didn't even have a fax machine. I had to go hunting around for a fax machine. And it was just on their side, they're worried about me coming and saying, you know, look, I wrote this 30 line file in Gradle and now I own the project or something. And so I think, yeah, the legal area is one that has kind of fallen behind the way the ecosystem is now.

Right, right. Well, and I mean, I've, you know, I've heard those lawyers say things to me as well. But the reality is that, like, one of the reasons why we have a foundation is because we have real lawyers that can tell us, like, the real risk profile here. And we also, you know, have IBM and Microsoft and these huge companies that also have lawyers that are interested in this. And so we end up with policies that everybody can agree to. But the opportunity cost of not getting a contribution because you have something like that is hard to quantify when the view of open source is that it's people that will jump through all these hoops that are highly motivated. And the reality is that actually we need to start

changing open source processes and policies and barriers to entry to accommodate a much more casual contribution base. So yeah, makes sense. Yeah. Awesome. Awesome. Yeah, it's great talking to you. Yeah, thank you so much. This interview is amazing. I think it really resonates with people. I mean, I personally, up until recently, I thought open source, various open source projects like Node.js, I would imagine it's five people in their basements around the world and just kind of hacking away at stuff. And it seems kind of very distant. The users and the contributors seem to

be just two completely different communities and what you see especially now is all of that kind of coming together and community managers like you helping to sort of organize that and sort of push that philosophical shift so it's very cool and thank you

for coming on the show no problem, anytime philosophical shift. So it's very cool. And thank you for coming on the show. No problem. Anytime.