Programming Throwdown - 146: RubyShield, Ruby Central, and Shopify with Mike Dalessio and Evan Phoenix

Episode Date: November 14, 2022

In this tour-de-force, Mike Dalessio – Engineering Director at Shopify – and Evan Phoenix – self-described “long-time Rubyist” – join us for a practical discussion of all things Ruby! Ruby is a beautiful language, and we're really excited to cover the history and present of this language with two experts. 00:01:03 Introductions00:01:49 Mike’s Ruby journey00:12:28 Evan’s own Ruby experience00:18:20 The pickaxe book00:20:34 Weird programming interests00:25:11 MINASWAN00:30:33 Language conferences00:36:38 Wrong answers on StackOverflow00:41:53 RubyCentral00:44:50 In-depth examination of Ruby00:47:57 How Shopify sticks to vanilla Rails00:50:28 A tale of two developers00:59:59 Bringing Ruby up to Python’s level01:04:48 Shopify’s largest app monolith01:11:12 Tuning the knobs01:18:01 How not to learn the hard way01:18:57 Opportunities at Shopify01:29:14 Working with the RubyShield program01:32:07 Rails for API servers01:33:21 Mike and Evan’s advice for listeners01:36:00 FarewellsResources mentioned in this episode:Links:RubyCentral:Website: https://rubycentral.org/RubyShield: https://rubycentral.org/ruby-shieldTwitter: https://twitter.com/rubycentralorgShopify:Website: https://www.shopify.com/Careers: https://www.shopify.com/careersDev Degree Program: https://devdegree.ca/pages/programHashiCorpWebsite: https://www.hashicorp.com/Careers: https://www.hashicorp.com/jobsMike Dalessio:Website: http://mike.daless.io/Twitter: https://twitter.com/flavorjonesEvan Phoenix:Website: https://github.com/evanphxTwitter: https://twitter.com/evanphxRubyConf 2022 (Nov. 29 – Dec. 1, 2022):Website: https://rubyconf.org/Other Episodes:Episode 47: RubyShow Link: https://www.programmingthrowdown.com/2015/10/episode-47-ruby.html References:“The Pickaxe Book” aka Programming Ruby: The Pragmatic Programmer’s Guide 2nd Edition:Amazon: https://www.amazon.com/Programming-Ruby-Pragmatic-Programmers-Second/dp/0974514055 If you’ve enjoyed this episode, you can listen to more on Programming Throwdown’s website: https://www.programmingthrowdown.com/ Reach out to us via email: [e-mail] You can also follow Programming Throwdown on Facebook | Apple Podcasts | Spotify | Player.FM  Join the discussion on our DiscordHelp support Programming Throwdown through our Patreon ★ Support this podcast on Patreon ★

Hey everybody. Ruby Central and Shopify with Mike D'Alessio and Evan Phoenix. Take it away, Jason. Hey, everybody. So 99 episodes ago, we covered Ruby back in 2015. If any of you kind of heard that episode, you might have been sort of shaking your fist in the car or at the stereo. Patrick and I have done a little bit of Ruby know we did our best to do it justice but i'm so thankful that here we are almost 100 episodes later the podcast has really kind of come a long way i'm really uh it's kind of wild just thinking about it but we have two amazing folks who know ruby way better than patrick and i will probably ever know in our whole lives. So we have Mike D'Alessio, engineering director at Shopify, and we have Evan Phoenix,

long time Rubyist on the show to talk about Ruby. Also talk about Ruby Shield, Ruby Central, how Ruby plays a big role at Shopify and plenty of other companies and, and really just dive deep into where the language has gone. So thanks so much, both of you, for coming on the show. It's great to be here. Thanks for having us. Absolutely. Happy to be here. Cool. So maybe we'll jump in just with a couple of backgrounds. Maybe, Mike, you want to go first

and tell us sort of what your journey has been like, what your experience with Ruby has been over the years, and what kind of is the journey, the path that took you to Shopify? Sure. I guess my journey to Ruby starts before I used Ruby. When I started my career, I was working in physics and then later financial services using Fortran and C, which are two of the least user-friendly languages you can imagine. And so by the time I hit like mid-career, you know, mid-2000s at this point, let's call it 2005, 2006, I started to use Ruby for a couple of hobby projects. And it just, it made sense to me, like something clicked, like Ruby is just, it's very easy to use. And you can tell that one of the principles behind the language from the creator maths is to make developers

happy. Like it definitely made me happy and something clicked in my brain and it became my primary language for writing like one-off scripts or even for doing bigger projects. I tried as hard as I could to use it for my next full-time job, which I did. And it's kind of been downhill ever since. I even got my start in open source with Ruby as well. I started contributing back upstream and some very friendly people in the Ruby community were very encouraging and it was painless. And I said, I guess I'll do this some more. And so now I've been using Ruby for something like 16 years and showing no signs of going back to Fortran at this point. Nice. So like, that's a really interesting thing. You fell in love with Ruby and then you said, you know, I would like to find a career where I can do Ruby.

And so my impression is that might be challenging in something like physics. And so you probably have to switch disciplines, right? So what was that like? And it must be a really interesting sort of set of calculus there to say, okay, I really love Ruby. I love physics. I can't really do both of these. I'm going to pick Ruby. And how did that whole thing go? Well, for me, I had already left financial services where you're expected to use C or C++ or, you know, on a bad day, Fortran. And I was already working at a small startup where I had the opportunity to play around with Python and PHP and Ruby. And like Ruby is kind of what I really like. I really rocked it and I really understood it well and it resonated with me.

So then my next job was at a small startup with a friend from college. And we had the opportunity to build a new app, essentially for, let's call it energy trading. It wasn't really trading, but it was a company that owns some power plants. You need to bid your energy into the market and do some light trading. And I was like, I'm going to write this in Rails and it's going to be great. And it all came together in about three weeks. I had built a replacement for their previous system that looked almost identical. And that was the opportunity for me was to kind of put a bunch of my skills together and write this new greenfield thing in Ruby at a new job at a startup where we

had a lot of flexibility around the tech stack. I think that was the key bit for me is Ruby was not a common language in 2007. And so I had to make my own opportunities in a way. We've come a long way since then. I think Ruby is very much enterprise-y at this point in time for a lot of companies. Yeah, that makes a ton of sense. We talked to a gentleman earlier who worked a lot with Dask and said something similar where they really wanted to use Dask and they found the only place at least local to them was a startup. And so it does seem like startups are a good place to have sort of more greenfield, which is kind of counterintuitive, right? You would think like big companies with zillions of dollars to burn would be the place to go to experiment with

new languages. But in practice, you find that they've built all the infrastructure around C. And so everyone's using C. And if you want to use Ruby, you're going to have to fight through like 17 layers of the organization or something. Yeah, I think that you're right on the money that it does work that way in large companies. And I think it's economies of scale. I think at some point, companies get to a size and scale where they say, listen, exceptions are really expensive to track and take care of and limit the flexibility, right? Like, oh, you want to do this project in Haskell?

Fantastic. You're one of three people who work for the company that know Haskell. And if you leave, then we're stuck with a Haskell project that nobody understands. We'll have to throw it away and start over. Like there's a lot of risks that big companies just don't want to take on new languages and new frameworks a lot of the time. Cool. That makes sense. So you're at this energy kind of management or energy trading startup. Did you go from there to Shopify or was there a string of opportunities in between? Oh yeah. There's a bunch of stuff in between. So from that startup, imagine it as I wrote some code for six months to try to get us

a beta version that we could build and sell. We were in startup mode. And then I needed to go actually pay my mortgage. So I got a day job and I was doing that at night. And my day job was at Pivotal Labs, which back in the day was a well-known agile consultancy that specialized in Ruby and Rails applications at the time. child welfare management, which is a really, really complex business domain. And Rails helped us focus on the business logic and not on the incidental complexity of building a large system. And that's when I really got sold on Rails as the easiest and fastest way to get something new up and running. I later became a client at another startup of Pivotal. Then I went back to Pivotal

again, where I ran their New York consulting business. And then finally, I worked on Cloud Foundry, which is like, it's like Heroku for enterprises, right? You want your developers to have that ease of pushing an app, but you have your own data center. Like, what do you do? You buy Cloud Foundry and you run it internally. And Cloud Foundry, as it turns out, was originally built by VMware in Ruby.

Weird coincidence. It was built in Ruby. But it turns out that Ruby is not great at doing things like handling, at least in 2016, it was not good at handling 10,000 simultaneous network connections, the kind of things that you need to run this big platform as a service. And so then we started selectively rewriting pieces in Go. But for the most part, there were big chunks of Cloud Foundry that today are still in Ruby. And so I ran engineering for Cloud Foundry for about four years too. And then I came to Shopify finally. Ah, nice. Very cool. So I'm trying to do the math here. So you joined Shopify four or five years ago? It was about two and a half years ago. Oh, two and a half years ago. Okay.

Two and a half years ago. Yeah. I was at pivotal for about 12 years total ah really long time really long time yeah that is awesome so pivotal is is is more like targeting developers different than so shopify is more of like a b2c or i guess it is a b2b but it but it has a different sort of customer base. And so was there any shearing effect going from Pivotal to Shopify? What was sort of the biggest differences there? So definitely the biggest differences were, you know, CloudFoundry, we were selling to like the Fortune 100 companies, right? We're selling to like huge financial institutions, huge retail institutions. And Shopify is selling a product to a lot of mom and pop store owners, especially during the pandemic. A lot of mom and pop store owners

who no longer could open their store and wanted to get online. And Shopify was a really easy way for someone who has inventory and a store and a business model to start selling online. That's what Shopify is really all about. It's trying to make in the same way that a great language will help a developer focus on the business logic that's important. Shopify helps a store owner focus on the business and not on the problems of accounting and how do I get connected to a payment gateway. If someone's not a software developer, they don't want to deal with that part of running an online store. And that's what Shopify takes care of.

Got it. That makes sense. So when you joined Shopify, there must be a legacy of Ruby at Shopify for it to really pique your interest. There is. So Toby is our CEO. He's only got one name. He's like Madonna. Toby, Toby Lutke. He is an old school Rubyist himself. And he, in fact, was one of the early collaborators on the Rails framework, which is the big web framework in the Ruby community. So he started Shopify. He was like the second person ever to use Rails, right? He and David Hansen were friends and he started using it to build Shopify.

And I think he feels very strongly, if you asked him, he would tell you that he feels strongly that Shopify was successful because he was able to move so quickly, right? He was able to get to production faster than his competitors. And there were a couple of instances that we have an internal podcast at Shopify where he talks about this. And he said that there were a couple of occasions where Shopify was close to going out of business. And the fact that he was able to ship new features and get new customers in a matter of weeks made a real fundamental difference in whether Shopify was going to make it through those tough times. And since then, we've, of course,

IPO'd and now we're a multi-billion dollar company. So it's all in the past. But I think he credits Ruby and Rails with a lot of Shopify success. And so we have a very strong opinion internally that that's the default stack we use. Cool. That makes a ton of sense. Yeah, I have not a counter example, but a correlated example where we were trying to build a while ago at another company. We were trying to build a like a new another company, we were trying to build a, like a new product, a new area for the company. And because so much was done in Java, they said,

we'll write the whole thing in Java. And I actually, and this is a bit of my personal, like anecdotal opinion here, take it with a grain of salt, but I feel like Java really caused the product to slow down, the product development. And part of it was not, it wasn't just that it was in Java, but it was also that it was hooked into all these other systems that were written in all sorts of different languages. And so they all had different APIs. Oh, there isn't a Java API for this backend. So now we're all just sitting on our hands. And I think that's one of the reasons why that project just couldn't get off the ground is that it couldn't move fast enough. So we'll

definitely talk about, you mentioned several times agility with Ruby and Rails. We'll definitely cover that, but we'll put a bookmark in it for now and switch gears and talk to Evan. Evan, why don't you tell us what your background is and what your sort of connection is with Ruby? Sure. So my story starts in the first tech bubble, the bursting of the first tech bubble. So I graduated from college in 2002, which was the uniformly known as the worst possible time to get a programming job. Uh, like every, every single place was firing. No place was hiring. I remember that same time I was in undergrad, graduated a little bit after you in 2004. And I remember my, uh, my dad basically shaking his head saying, I told you,

you should have got a math degree. You went and got a CS degree. Look at you now. I mean, you were right in the end, but at that time, it was tough. So it was 2002. I had just graduated. I moved to Seattle. My then girlfriend, now wife, and I moved to Seattle. She was starting her master's program. And I'm not a great student, I had no interest in continuing on with higher education. So I was like, I'm just gonna figure it out. And so I tried to get a job couldn't get a job, ended up getting a job with a local ISP. It was a three person ISP. And I basically put sat like 802.11 not be so pre B standard 802.11 dishes on people's roofs for this ISP because it was a wireless ISP. And so I would spend afternoons on roofs, retirement buildings on roofs,

lots of roofs. I've seen so many of the roofs of Seattle. I should have started a scrapbook in retrospect, so many roofs. So I'm at this job, and I don't want to be do I wanted to be a programmer, like that was what I had done. I'd actually done open source in college. I had worked on some programming language stuff in college, because it was really interesting to me. But now I didn't have any opportunity for that. But when I moved to Seattle, I was very active in a specific IRC channel from college and so and one of the people in the IRC channel actually was like hey I'm writing this really interesting thing and I found he he was also in Seattle coincidentally this guy even though we'd been talking for years and uh he

was like hey I'm writing this thing and um it's in this programming language called Ruby and I was like hi I'd never heard of that and so I went to go look at it and it took me about a couple of weeks before I was like, because I had done Perl and PHP almost exclusive, well, and C++ and C at that point in time, but on the sort of Ruby-ish side of things, PHP and Perl. And so it took me a while to get the feel for Ruby. But once I did, I was like, oh, this is great. I like this a lot better than the stuff for Ruby. But once I did, I was like, oh, this is great. I like this

a lot better than the stuff that I've been doing before. It's much more, it's easy to express what I want. I'm not fighting with pearl sigils, which is a little symbol you put at the beginning all the time. I liked it a lot. I ended up getting involved with Ruby and then not seeing that RubyConf, which was like the Ruby conference that started in 2000, I actually missed it by a week. It was in Seattle. It was down the block from me. Didn't go because I didn't realize. I literally was just like, oh yeah, Ruby, that's a cool thing. Oh, the conference was here last week. It was pretty amusing. But I did get involved with some other Ruby people in Seattle, Ryan Davis and Eric Hodel. And we started the first, it's hard to say if it's the first, but effectively people used to call it the first Ruby user group, Seattle RB.

And it was just the three of us. We used to meet at Ryan's work and just putz around and do random talk about things. We ended up getting, like I remember one time, we got a review copy of the Pickaxe book, which is like the canonical, the original reference book for Ruby that Dave Thomas wrote. And we did a review of that to set Dave notes because I think Ryan knew him at the time. Anyway, so that's how

it started so you were you're working for this uh ISP you're you're mounting uh dishes wi-fi dishes on roofs and but but uh but you were involved in a lot of these kind of uh social groups and and and sort of you know building up this Ruby ecosystem. And yeah, that's actually a testament, you know, I think where there's so many companies talking about this looming recession. I don't know what's going to happen. Nobody really does. But something to sort of foreshadow or some foresight based on your experience is to, you know, even if the job market does contract to get difficult to there's always

so much that you can do, you know, as an individual, there's so many different conventions you can go to people you can meet. And that is going to be more and more important. We sort of, you know, go through this, like kind of small lull here. Yeah, I mean, recessions affect the economies of monetary policy, not people's enthusiasm or interest. Right. Right. Yeah. Yeah. Like, yeah, at the time it was impossible to get a Ruby job. Well, at the time, Ruby jobs did not exist. It was not a thing. Matt's had only begun Ruby in 1996 and it didn't, it only lived on his laptop for the first couple of years anyway.

And so we're talking 2002 ish timeframe, maybe 2003, a little bit now. And Dave Thomas, the guy who wrote the pickaxe book was, is kind of credited with bringing Ruby to the English speaking world. Matt's is Japanese. Dave has a great story. I can't remember the exact details, so I'll probably butcher it. But effectively, he saw Ruby, he saw some of the original documentation, he was getting into technical book publishing, and he wrote the first English version of the pickaxe book, the root, we call it the pickaxe books, it has a pickaxe on the

cover. And so he wrote the original version of that that basically was the only guide to Ruby that was written in Japanese. And I think, so I'll fast forward a little bit. I ended up stumbling into another job accidentally, working as the Windows IT administrator, which was better than putting dishes on roofs. So I went to go do that. And then I, but I still wanted to be programming. And so I started just it was a startup in Seattle, I started attending all the engineering meetings, they started, they had a meeting where they were like, hey, like, we want to we want to redo the back end for this thing, because it's not working very well. It's really cobbled together,

we want to actually use like, basically it was like sort of a Linux system administration slash programming project. And I was like, I am very interested in this. If no one else has, I've actually done this before because I'd done it in college. I will, I'd be happy to take this on. And they're like, sure. I was, again, I was still the Windows IT admin. And so I was doing the Windows IT administration, keeping the mail server up, making sure people could log in, and writing this new backend. And when I was writing this new backend, I was like, this is sort of what you were talking about earlier. Nobody was telling me anything. They were just, get it done.

We just need a thing. And I was like, great. I'm going to write it in Ruby because that was way more interesting than writing it in anything else at the time. So I started writing it in anything else at the time. So I started writing it in Ruby. I wrote this amazing gateway so that PHP could call Ruby methods on the same machine because all the front end people were writing PHP. It was great. It was an insane system. PHP, Ruby, Interop. Yeah. It's somewhere out there. It might not be on the internet even. Anyway, so I worked there for a while. Ended up leaving a couple of years. Oh, like that, they got built up.

We ended up doing a lot of Ruby at that company. I ended up leaving and then going to another small startup for about six months, basically writing a Rails app. And then in 2006, that was in 2000. That was like the beginning of 2006. In 2007, I was still living in Seattle, it's gonna move down to LA, I ended up going to RailsConf that year. This is another sort of diatribe, we'll keep it short, which is, I have weird programming interests. I had been writing a new version of Ruby, the programming language, the interpreter. So Ruby, as a as any

kind of dynamic language is just a program that reads other programs and runs them, right? It's like a chef that's running a recipe, but the program is the chef. And so I had all these interests in how to make Ruby better, and I had started writing my own version of the actual interpreter, the engine to run Ruby programs and i had been doing that for a year and a half two years at that point and i had i had a thing that was interesting um i ended up giving a little talk about it at rails conf which is in portland that year i drove down from seattle to portland to talk about it and the cto of engine, who was an early Rails hosting company, saw my talk and he's like, I think your thing's cool. Tom Marnini is his name, CTO.

I think your thing is really cool. Would you be interested in coming and working for Engine Yard? And we'll let you work on this thing. And I was like, this is the dream. Oh, that's amazing. This is the dream, right? Like someone wants to pay me to work on my open source project. So I said yes as fast as I could. And then I did that for five years. I just worked on my own, well, an open source Ruby implementation for about five years called Rubinius. And yeah, that was super intense. There's lots of stories about that.

Where did Rubinius go? Like did parts of it get folded into Ruby or are folks still using it? Well, it had a lot of contributions. I would say that a big portion of the stuff that we wrote, it lives out on the internet, but it's probably not in heavy usage. Probably there's a couple of people who still use it for random things. What ended up happening was I started writing Rubinius when Ruby was at version 1.8.2. That was in 2006. And at that time, Ruby was actually moving pretty slowly in terms of the API, the functionality, all that kind of stuff. And so it made it easy if I was like somebody's trying to come in and being like, I want to implement this API, which is just the program languages API. And it was about a year after that or so that the mats and the Ruby core team really started to speed things up.

And it only got faster and faster over the next couple of years. And it's hard. It's very hard to write a thing that's chasing something else and you don't have any control over it. You're just chasing this other thing. And after about five years of working on it, I was personally like, I want to be more in control of my own destiny. Like every time, like every day I was coming in and being like, what crazy weird API do I have to implement today that I don't want to? And that's not fun as a job. I ended up leaving there and going to LivingSocial. But yeah, that was that bit. Oh, interesting. I think there's one other thing I want to throw in here

while we're talking about Rubinius. If you look at Ruby on the decades-long timescale, one of the, I think, most important outcomes from the Rubinius project was actually RubySpec, which is the specification for Ruby. It was an executable, it was tests, right? They were an executable specification for Ruby because the original Ruby interpreter, it was just like, well, it behaves how it behaves. Like it's the only Ruby interpreter. And so there were a lot of edge cases that were kind of undefined behavior. And as part of the Rubinius project, that Ruby spec project is actually still alive and kicking today and unifies all the Ruby interpretations because there's like JRuby,

which is built on the JVM, and there's TruffleRuby, which is built on GraalVM, as well as Ruby and Rubinius and a few others that are all using Ruby spec as the de facto standard for the Ruby language now. I think that that is a really important outcome as well. Yeah, that totally makes sense. I think the you know, the parallel that comes to my mind is io.js, right? So Node had some deficiencies. So I think it was maybe the original Node author. I don't actually remember. I might be getting that wrong. But somebody wrote EO. And long story short, EO and Node merged later on. And so a lot of the best parts of EO made it into Node. And so I think, yeah, it sounds like something similar that

happened here where as a consequence of Rubinius, you have this specification for the language, and that created a huge harmony among all these different interpreters and push the language forward. Yeah. One of the most interesting things, the Ruby community is a, it's a by-product of, there's a, there's a saying in the Ruby community called Minaswan, which is maths is nice. So we are nice. And that there's a big, there's a big, it seems like a thing that people wouldn't actually say, but people, people mention it all the time. It's really part of the ethos of the Ruby community. The Ruby community has spread its wings as it's gotten bigger in time. And I think one of the most interesting things is that, this is my skewed perception, but my perception is that the Ruby community is one of the first communities that really took TDD, test-driven development, to heart. It existed mostly from

Kent Beck in the Smalltalk world, but there isn't that many Smalltalk developers. And a lot of the Smalltalk developers ended up in Ruby or were influenced into Ruby at various times. And I remember being exposed to TDD in 2002, 2003, when nobody was doing it. And that was a big impact on Rubinius writing RubySpec. Brian Shirai, who was like the second or third person to work on Rubinius with me, started that specifically because it was like, hey, we got to know what we're doing. We got to write some tests. We got to write specification for this thing.

And still to this day has a huge impact. So Ruby spec ended up being adopted into the mainline Ruby interpreter. They still write those specs. One of the big things about Rubinius was I tried to write all of or as much of the Ruby API in Ruby itself. So you could go and look at the implementation of array and it would be written in ruby for instance that code made its way over to truffle ruby which is a very specialized implementation of ruby and has evolved since then they have they've edited it's they you know they're perfectly much more owned by them at this point but um it's kind of seeded itself in different ways very cool so after Rubinius, at that point,

what's the sort of link between that and and Ruby gems, which is, which is what your focus is on now? What's the path there? Yeah, so I had been going, I started going to Ruby conferences in 2005, which was in San Diego, I got to, I got to watch the first release of Rails be released. They released it at RubyConf, which is just Toby and David in the front of the room releasing it. It was very interesting. So I ended up starting going to Ruby conferences then. And they were great. I was in the community. I was super interested in it. And I kind of just kept going. As my second job or whatever in this story showed, I fall into system administration jobs fairly regularly. Tasks maybe is probably the right word, rather than actual jobs. And in 2011, there was a thing called RubyForge, which was a hosting platform that was meant to,

this is pre-GitHub. So it started pre-GitHub. It was basically a place for people to store their Ruby code, right? That was really what it was for. And it was sort of an alternative to SourceForge, which some of your audiences heard of and some of your audiences has not. It was a sort of a precursor for where to store your code on the internet. Yeah, SourceForge, if you've ever needed an old library and you end up clicking on 14 ads and you don't actually get the zip file of the library, that was SourceForge. That was SourceForge, yeah. Oh, man, you could do a three-hour thing about the evolution of SourceForge as well. So there was RubyForge, which really just ran.

There was a package called GForge, which was just like you could run your own SourceForge-like thing. And so there was a thing called a couple of Ruby developers ran a thing called Ruby forge, just ruby forge.org, people would upload their things. And then in 2007, or so I'd have to go back and check if it was 2006, seven or eight, I can't remember a couple of Ruby developers, Chad Fowler and David Black, who are all directors at Ruby Central, which is the nonprofit that puts on the conference, start basically at a conference. We're like, we need our own like Java jar like thing. So let's start, let's write one. And so they basically wrote Ruby gems. The first version was a couple of weeks, maybe probably the first version was probably

at the conference. So within a couple of days, and then they wired in the ability to start to to handle Ruby gems inside RubyForge. So if you uploaded a gem file to RubyForge, it would be available for you to do gem install. That was the first version of RubyGems. So fast forward again back to 2011, I got wind that they were still running RubyForge. Well, I knew they were still running RubyForge and that there was only one person who was this administrator for this whole system. And I was like, that's not cool. That's not fair. And I was like, that's not I'm going to reach out and I'm going to say, like, I'm happy to help you out run this thing. And so I reached out just to say, like, do you need to have another hand, another person who can basically help you on call, because the guy who was running it is a wonderful guy, but it was not fair to him to basically be doing this whole thing. So that's how I got involved with Ruby gems. Originally, it was basically I'm going to help out run. Well, a little bit, I got involved with Ruby, Ruby gems software a little bit earlier

than that. But the bulk of it was really the stuff we're talking about today is really helping out run the server, debug some crazy things, and then, yeah, that was how it began. So what is it like going to a language conference? So let's say you're a college student. You really love, I'm just going to pick a language. You really love Ruby. Maybe you've made pick a language, you really love Ruby, maybe you've made a few commits to the sort of core, you know, Ruby interpreter or other parts

of the ecosystem. But you just love programming in Ruby, you know, is like, what would someone like that get out of a conference? How would you recommend getting the most out of a conference if it's your first time? What's that whole thing like? I've been to, personally, I've been to a lot of conferences, but admittedly, I've never been to a programming language conference. It's on my bucket list. But you kind of described to us all what that is like and what you would expect to see there and how to really have an effective time there. Let me take a whack at answering this real quick. So my strong opinion here is that there's a spectrum of conferences and there's like some conferences are small. Like I used to organize, co-organize the Gotham Ruby conference

in New York City, which was one day, one track, very intimate conference. So everyone had the same shared experience. You show up, you sit next to somebody, you get to know them, you go to lunch, you come back, there's some more talks, then we'd go on a boat and have dinner. And it was great. I think that for me was my introduction to software language conference. It helps you build a local network of people, and there's usually a really great mix of introductory talks as well as some more advanced talks. And then I think once you get comfortable and you go

to one or two of those, then maybe you want to say, okay, I'm going to get on a plane, and I'm going to go to a bigger conference, and maybe I go to a three-day Ruby conference in New Orleans, which was my big experience at my first big, a bigger conference. And maybe I go to like a three-day Ruby conference in New Orleans, which was like my big experience at my first big, huge software conference. And like, that's like a really big deal.

And then the social network that you've built locally can help you there. But again, there's usually a great mix of both introductory and more advanced talks at all of these conferences. So you can kind of pick and choose which ones you're interested in and want to pay attention to. And you get to talk to the speakers afterwards. And it's as much about the social network you're building as it is about what you're learning inside any individual talk for me.

I don't know, Evan, what's your experience been? Yeah, I, the background, I didn't give this at the beginning, but like I helped run RubyConf and RailsConf for about 10 years. And so I have like, there's three different styles of programming language event, I recall that. The first, like Mike was talking about, is a seminar. That's really the way to think about it, right? Which is basically, you're going to show up, you're going to all sit in the same room, you're going to all watch the same talks, you're going to find out about interesting things that you didn't know that you were interested in, or and maybe they're going

to be over your head, but then you're going to go sit with people at lunch. And you're going to say, can you explain this to me a little bit more, because it seems interesting, and I don't really get it. And you're going to have some some nice person's going to say, like, I'd be happy to, you explain to them, now your friends, now when you get on the boat later on, maybe you meet their friends. And that's a seminar in general, right? It doesn't even have to be programming language specific. That's how all seminars generally work, which is that everybody sees the same thing. Everybody talks about the same things.

And then you all sort of bond over that and you learn, right? So when you go home, you go like, I found out about testing, I found out about asynchronous programming, whatever it is, and you want to take that back with you and kind of let that percolate, figure out where it's going to come out in your in your day to day job. Second one is programming language conference, like just regular conference, which is like multi track conference, RubyConf is an example of this, you have a lot of different you have like three or four two three or four talks going on at the same time you have to pick

what you're going to go to there's lunch there's a keynote talk there's that kind of thing it's it's definitely a little more overwhelming people have to figure out where do i what do i go to right and i would always people always ask me that at those conferences and i would say like just if you don't know what to go to, just walk into a room, just wander into one of those rooms and sit down in the back and be like, what are we talking about? Right. Those are some of the best experiences you're going to have is if you never knew what you

were looking for until you sat in the back of the room and you found out, found somebody talking about Ruby reflection or something like that. So that's not, that's number two. Number three is sort of trade show. RailsConf kind of fits into this, this one, which is more like a trade show, which is really where you have an actual exhibit floor, you have exhibitors, you have people who are going to show you interesting things, you have sponsors, you can solve talks as well. Trade shows always have like continuing education. But there are a lot of places where

people are looking for jobs. People are showing off their companies. People are networking. That is more of a trade show and they all have their place. They're all sort of on the level of overwhelming. Those are ratcheting up the overwhelmingness on that thing. But yeah, there's sort of those three. Those are the three kind of

blends. So cool. That makes sense. Yeah. If any of your listeners are worried about whether the conference is right for them, like, I would say if you think it's for you, it is for you. And you'll be surprised when you show up and there are 200 other people that have common interests that you can chat to about. So even if you're introverted, or you're shy, or you're not really sure, like what you're going to talk about, generally, folks are going to turn up because they're interested in the same things as you. And it's usually a very welcoming experience I found. Yeah, definitely. You'd be surprised to if you have, you might be sitting at home very introverted. But if you have a topic that's really passionate to you, you'd be amazed at how quickly you will become an extrovert when you're surrounded by other people who have the same interests. I saw this joke the other day where someone said when they have trouble with their CS homework, they ask a question on Stack Overflow.

And then they have a second account, a dummy account where they give the wrong answer. And then that's what they need to do to get like hundreds of right answers. So, you know, there's so many things where when you dive into the details, there's, you know, it comes from, there's sort of differing opinions and not really any right answer. And so you're going to come with your opinion that you've calcified, you've spent a lot of time and energy on, you're going to find someone with a totally different opinion. And the two of you are going to have a really engaging discussion. And at the end of it, you probably get to a place of even better balance than when you started. So yeah, that was

one thing I found. I was pretty introverted, I was going to a bunch of machine learning conferences, but I found when I was at the conference, then it was like, Oh, yeah, I really want to explain this thing. And here's my take on it. And all of a sudden, like all these words start coming out. I wanted to say one thing, which is that I want to highlight what effectively I don't know what other people call it, but we call it the hallway track, which is the thing that happens when you're not in the sessions, but is during the conference time. So it's not at lunch or whatever, right? Because at Ruby conferences, at least the hallway track is as just as valuable as any other time that you will be there. Mike and I, I think,

I think our first time meeting was in Orlando outside the ballroom. Basically, this is RubyConf 2006, I'm going to say. It was in Orlando. It was in a hotel where we couldn't leave, basically. It was in kind of nowheresville in Orlando. We're all sort of sequestered in this hotel, but it was a nice hotel. And we all, the hallway track was amazing because there was like these U-shaped couches where there'd ended up, there would end up being like 12 people sitting in a U-shape, one of these U-shapes,

some on the floor, some on the couches working on things. I think, I don't know if you're working on Nocaguiri or H-Percat at that point in time, but, and then, you know, yeah, we would talk. Like, I remember debugging WebXML issues with you and Aaron at that point in time. Like, you know, does anybody know how this thing works? And someone would be like, oh, I've dealt with that before. And you kind of just do that dance, right? And then you find

that, like, oh, I didn't realize that other people were also interested in this or whatever, right? Something else I want to call out, Evan, I mean, I don't know if you have these numbers handy, but at every RailsConf or RubyConf I've been to, during the initial keynote, you ask, for how many people is this your first RubyConf?

And like half the room raises their hand usually, right? Yeah, so it's always half the room. Crazy as that may be, it's always half the room. I mean, I'm up on stage when I ask this question, so my perspective is a little skewed. But it's the most interesting thing that it's half the room. And I think for the audience who are listening to this and thinking, like, you know, is Rails something that I, like, are there people still picking it up? Is it a thing? It's very much still a thing.

And there are so many people who are just learning Rails for the first time every single year. It's learned in boot camps. It's done in all kinds of different places at small scale, at large scale. And the fact that it's 50% at RailsConf is so important to me because of what it says is that the community is always changing. It's always vibrant. It's not the same 500 people who show up every single year to talk about the same thing they did last year. You know, we talk about that when we would work on the programming for, you know, the scheduling for what talks we're going to have. We always think, should we have this talk again? And I

have to remind people, 50% of this conference will be brand new. So you can have the same talk every single year. And, and people would 50% of people would never have heard it before. Yeah, that's, that's wild. So, so you're, you're helping the support on this, I think it was a Ruby project. And, and was that was that Ruby Central? Or is Ruby Central something in the future? Yeah, so Ruby Central is an actual nonprofit set up in the United States that maintains. It was set up originally actually to run the conference, to run RubyConf. They needed an entity that could basically take the insurance load, if you will, of running a conference. And that was how it started. And then after I started helping out with RubyForge and rubyjumps.org, I was asked by the then directors of Ruby Central if I wanted to be

on the board if I wanted to help out with with Ruby Central at that point. And I was like, yeah, I do. Because I want to make sure that stuff actually happens. So I, yeah, I started helping out with on with rubygems.org as a board member, and then started helping out with the conferences. We almost tanked the whole thing at RailsConf 2012. That's a longer story than this podcast is available for. But I'm happy to tell it as some future junction. It's great. It has to deal with eating pot pies for three days. It's great. It's a longer story than this podcast is available for, but I'm happy to tell it at some future junction. It's great. It has to deal with eating pot pies for three days. It's great.

It's a great story. But, yeah, so I started getting involved then. And Ruby Central is the entity. So Ruby Central exists. There's directors. They have staff now. They didn't have staff back then, which is why we almost tanked it. But there's staff. which is why we almost tanked it. But their staff and it basically maintains

and organizes the RubyConf and RailsConf is an entity that can just basically, all it exists to do is improve the Ruby language and their community. And so it is the entity that pays for rubygems.org. It pays all those bills. It manages all of the things related to that. So it sort of is the owner of a lot of community resource.

So yeah, okay. So there's, I went to SciPyCon this year, which is, I guess, the closest thing I've been to a language conference. But it was an entity, and I'm totally drawing a blank on the entity that runs it. But I had a chance to talk to some of the directors and it was wild was wild i mean basically

they that same entity also you know make sure that numpy scipy dask all these core languages have sort of a healthy ecosystem you know one thing a lot of people take for granted is you know if you file if you get you know an equivalent of of internal compiler error or internal interpreter error at Ruby, and you file an issue on GitHub, someone has to be there to catch that. And so that's a huge amount of a tragedy of the commons where everyone is kind of taking, but there isn't really somebody who can contribute and contribute in a really organized, focused way. Yeah. One thing I wanted to say about Ruby Central as well, because I think that this has been a misunderstanding over the years. It's fine because we didn't correct it. Ruby Central doesn't really make money. It's a nonprofit. It exists basically to just put money back into the community. So when we run a conference, it's really just to pay the bills

for rubygems.org. So we always tell people, how can I contribute to rubygems.org? I say, go to one of the conferences, buy a ticket. Prior to 2012, RailsConf was actually run by O'Reilly, the book publisher. And so and that was run kind of O'Reilly ran it as a for profit entity. Right. Because that was what they were in. That's why they run conferences is for profit. Ruby Central took it over. We continue to run it. But we were like, hey, now we can tell you where this money goes. It goes it's for you. We we were like, hey, now we can tell you where this money goes. It's for you. We basically take in the money, but we basically give it back to you in the form of these community

resources. So there's not some big corporate entity where they're just taking in profits to buy jets or something like that. It literally just goes in there to basically be pumped back out into the community yeah totally that makes sense so cool yeah this would be a good time maybe to talk about about ruby at a high level my my first interaction with ruby was through um this this uh software called jekyll where you could spin up a static website which I needed for a GitHub project and the thing that I was really impressed with with with Ruby and Rails was how easy it was to switch databases and and things like that so I had I had a development version which was using a SQLite database and then I had a production version that was using some type of hosted, I think it was like a hosted MySQL database. And all of that

back in the, when I was writing data, web services in Python, I would have to build all of that from scratch. I'd have to say, okay, check some environment variable. If it's this, then use the development database. And oh, you know, I don't have the, or I don't have the MySQL library. So I have to go pull that in. It was a lot of manual effort. With Ruby on Rails, you kind of get all of that for free. That was the thing that I really noticed. So yeah, maybe, you know, with your kind of expertise, you know, what kind of separates Ruby on Rails from from um you know what kind of makes it distinct so i like to think about rails as being like really well vertically integrated pieces of

software right like you can come at building a website from like oh i'll pick my own like artisanal database adapter and then i'll figure out what how to render views then I'll figure out how to render views. And what Rails did and continues to do is put together pretty strong opinions. This is how you want to interact with the database. This is how you want to render your views. This is how you do your controller. This is how you do all of your models. And this is how you unit test all of your models. And those conventions are really liberating, I think, as a developer. I like not having to think about the small details, like, well, what directory do I stick this file in when I do the thing? Like you just say, I need a new model and all of these things like self inflate for you. And I feel like over the years, other web

frameworks have borrowed a lot of those same like strong conventions and opinions like Django. If you're if you're running an app in Django, I think Django has borrowed a lot of stuff from Rails over the years. So for me, at least, again, like if you're going for, I want to get something up and running on the internet as quickly as possible, that's going to set me up to continue

to add new functionality and continue at this really high velocity over time. Like Rails really sets you up as a software developer to do all of that from day one out of the box. And if you want to color outside the lines a little bit, Rails does allow you to go ahead and configure it however you want. And there are a lot of really interesting plugins and alternatives for parts of Rails. But for the most part, I think Vanilla Rails is a great way to run a business.

This is what Shopify does. And part of what my team does is make sure that Shopify is running as vanilla a flavor of Rails as we possibly can. And if we need something special, we try to push that back upstream into open source so that everybody can benefit. And I think if you look at other companies like GitHub and Airbnb, they've made contributions back upstream to Rails over the years as well to make it a better framework. And now I'm going to like grind my open source axe for you a little bit and just be like,

this is what makes open source so wonderful is that these companies that are using the software in anger to do something real are all motivated to contribute back to keep that flywheel going of making the software great and attracting new developers to the community. Yeah, I think DHH, David Hannon, the originator of Rails, he said it a couple of times, I'll paraphrase, which is basically that having constraints helps you be more creative.

And that philosophy also is true in just how you set up your stack, right? So having all those opinions also means that there's more constraints like, oh, I have to use this, I have to do this, I have to use this one piece. Now, yes, that basically means that like, if I don't want to use it that way, maybe that's annoying. But the flip side of having those constraints and assuming that they're going to be there is that all of those things work really well in concert together. And they allow you to basically say like, okay, maybe I didn't want to spend or most people don't want to spend, you know, months trying to just figure out how to write the best SQL query. Great. You don't have to do that. You can use active record. And

that as a part of the stack as constraint, but it also adds an incredible amount of commonality. So that as a lot of developers will know, when you move from one project to another, so you move between companies, right? You move between companies and they're both writing web apps means almost nothing about the idea that you will be able to pick up the new company's app, right? It could be a different language. It could be, even if they're both JavaScript, they could be completely different, like off the wall different, right? If you move between two companies and both of them say, hey, we're both writing Rails, you'll probably be productive on your first day at the new company.

You'll probably go in and be like, okay, did you do this? Did you make this decision or this decision? Oh, you made this one? Cool. Got it. And then you're often off to the races, right? And that has such power in being able to have people be productive, fulfill what they need, and also be expressive at the same time. I love the talk about sort of the opinionated framework and being able to build something that lasts. I think it's such an important piece. And it's also a trap. I'll give kind of a quick tale of two developers. I have two buddies in college.

The three of us were eating pizza and talking about video games. And the two of them both wanted to become video game developers. And I was just sitting there eating pizza. It's not my thing. developers, and I was just sitting there eating pizza. But one of them said, I want to use Unity, because even back in, I guess it was like 2005, Unity was around, and it was an easy way to get started. And the other one was saying, no, I can't do this and this and this in Unity, and I can't do all these really specific things. I think we should write everything from scratch, like build our own game engine, our own everything.

And I actually don't know what happened to that second guy. So I don't know. Maybe he made his own game engine. I don't know. But the first one actually started a game indie game studio and did really well. And so I think this trap that we fall into as creative people is we want to have the most creative medium. And the most creative medium is arguably like, I don't know, assembly or something, right? Where you can do absolutely everything. But the problem is that actually when you're getting started is when you need to be as

least creative as possible with the particular paradigm you're working on. So if you've never built a website before, you should focus on having a nice end product and being able to be creative there by iterating quickly and not spend a lot of time saying, oh, how am I going to, you know, like, what if I need to use this really esoteric graph database, right? So don't worry about that stuff, just get the thing up and running. And the opinionated thing, whether you're talking about Next, or whether you're talking about Rails, it's going to put really polished ivory handcuffs on you. And in the beginning, you're going to say, oh, I'm getting all these weird errors. And, you know, oh, it's not letting me, you know,

put everything in the same folder, all the source code in the same folder, because at least for next, you know, the folder, the directory structure determines the API. So you find all these road blocks. But then once you've figured out how to make your thing fit in that paradigm, all of a sudden, you get all of these benefits that you are, that were outside of your, your conscious coherence. So all of a sudden you realize, Oh, now I see how, you know, if I wanted to add a piece, I can do that without, without breaking everything else. Right. So yeah, I think this is a trap I've personally have fallen into a lot as well, where I say, Oh, this, this piece of software doesn't easily let me do X. So I'm going to write my own thing. And then sure enough, your own thing never actually gets done.

I was just going to say, I have been through this many times. And the way that I like to think of programming specifically is it's an artistic tradecraft, right? Like you can just like plumbing, you can pick up programming. You can start doing it. You can learn from other people. You can read a book. You can just ask somebody. You can be an apprentice. You can do all those things and you can be the best programmer in the world. That definition makes it a tradecraft. But so is watercolor painting. Watercolor painting is also a tradecraft. You just sit down and you start watercolor painting. You read a book, you look at it, how somebody did it. You just start doing it, right? Programming is somewhere between plumbing and watercolor, right? And one of the most interesting things about, say, watercolor is that when you decide to do that, it is so constricting. You're like, oh my gosh, I can only make colors like this. I've only got these kinds

of paints. It only works on this kind of paper, right? But think of the artistic creativity that's come out of just doing watercolors, right? The world over, right? That constraint breeds creativity in other ways that are potentially much more interesting than just being like, oh, well, I need to express myself. And so I need to spend two years making my own paper. Great. Maybe you will achieve something great. But more likely, if you sat down and just learned how to watercolor on normal paint for two years, you'd be able to express yourself potentially better than saying, I need the best paper in the world. Thank you for saying that, Evan. I agree 100%. There's a wonderful article, a blog post from 2015 by Dan McKinley, who if you all just, if you're listeners, Google for choose boring technology,

you will find this essay. And he introduces this really interesting concept of innovation tokens. Like if you're building a thing, you should say, okay, I have two tokens that I can use for some innovative technology and everything else I do has to be boring. And this forces you to make the choice about where do you want to take that risk? Where do you want to spend that time to rewrite a gaming engine or go grind your own watercolor paint color or make your own paper? And think about what is the core competency that you're going for? Like if you want to start a new gaming company that's predicated on faster frame rates, maybe you want to go build your own gaming engine. Yes. But if you want to build a gaming company built around like super fun stuff, maybe the gaming engine is a distraction

and you should use something that's standard off the shelf and put your time and effort into making that game super fun. And I think about web development the same way. Like I could write my own like socket handler from scratch, or I could just use like a load balancer off the shelf and just, you know, there's, there's a ton of trade-offs that you can make along the way and people should feel free to make those trade-offs, but like choose boring where you don't actually need a competitive advantage is how I would think about it. Yeah, that makes sense. It's actually a paradox, right? Because if you can make something fit, I'm thinking about the watercolor example. You see examples where people take a style that's very difficult to express in watercolor or in any other medium and they make it happen

and people really appreciate that. It's sort of like when you see these pixel art, your gigantic landscapes and pixel art and all of that. And so, and so, um, uh, you know, that could be sort of part of the challenge instead of saying, well, I'm going to write everything from scratch. And, uh, that way I could be as creative as possible, say, I'm going to make this work in this framework. And that actually is going to force me to be as creative as possible. You know, ostensibly, Ruby is a language, Rails is a set of frameworks, but, you know, like, where is sort of the gap there? What other things is Ruby used for?

Other than Rails, like, what kind of captures a lot of the Ruby market share? And how do we sort of differentiate these two things for people? Yeah, I think if we were having this conversation six years ago, I would probably say DevOps was also in that group, right? So famously, Puppet and Chef are both written in Ruby. Oh, I didn't know that.

Yeah. So if you go to write Chef cookbooks, you write Ruby. If you go write Puppet thingamabobs, I can't remember what the noun is for puppet. Plans, I don't know. Yeah, you go to write puppet Pinocchios, then you actually write them in Puppets specific DSL, but that DSL is actually parsed in Ruby. So if you wanted to extend Puppet, do different things, you actually write Ruby on the backend. So that was big. Those things were both that were born out of this time, this interesting time in the programming community. The zeitgeist of Ruby circa 2010, 2009, around that time where people were like,

hey, let's write everything in Ruby. And DevOps was coming up and cloud was getting big and we hadn't really hit containers yet. And that was really the era of those tools. Those tools still exist. People still use them every day. I would say that they're probably on the down slope a little bit right now because cloud and containerization

have given people different ways of solving the problems that those same things solved. And so you don't see them as used as much anymore. I think Rails is still the 800-pound gorilla in the Ruby community. I think there are other people. You can write, I mean, it's Turing complete.

You can certainly write other programs in it. People write all kinds of other services, non-Rails, web services, internal services, all kinds of stuff like that in Ruby as well. But I think that service-based applications, be it web or otherwise, are probably the lion's share of what we see in Ruby today. What do you think, Mike? Yeah, I would agree. I think there's a healthy number of people who are pushing Ruby towards being a better platform for big data analysis. The big Ruby conference in Japan happened last week, Ruby Kaigi. There's been a series of talks over the last few years

about how to integrate Ruby with Apache Arrow and how to get low latency data access and how to do OLTP and OLAP. And I think that's really, really interesting because I think Python has been the dominating language for a lot of that. And there's no reason Ruby can't be equivalent, except that we don't have a lot of the same libraries that the Python community has invested in over the years. And so I do see some hints there of like, well, what are the interesting things that Ruby hasn't traditionally been used for, like big data and pushing in that direction,

which I really like. Yeah, that makes sense. There was a big effort, Swift Torch, it was this effort to do tensor processing in Swift. And that, I think it was Google actually was pushing that, even though Swift is an Apple thing. I think Google is really pushing this like Swift TensorFlow and Swift Torch. I don't know what happened to that, but I think there is a space for other languages to be in that domain. I think there's a lot of times you need to do tensor processing, you know, no matter where you are. So imagine if you are trying to do some analytics on, you know, within your web service. You're seeing a lot of this now where, you know, as the third party cookies and other things are getting blocked, you know, people have to push the analytics to the server side. So it used to be just a bit of background used to be, you know, anytime you would, well, depending on how they set up the website,

but maybe every time you click on something, you know, it would send a little packet of information over to Google that would then make its way into that company. So for example, you're running a website for your pizza place, and you want to know how many times are people clicking on your menu bar. So you would set up Google Analytics. Every time someone clicks on your menu bar, it fires a little packet of energy to Google. Google accumulates all these and sends it to you. So now if someone's using Brave or using, I think the new Safari has this, those are all blocked. So, you know, if I'm talking to, you know, PizzaHut.com, I can't actually, my browser will actually block packets going to Google if I'm supposed to be talking to Pizza Hut.

And so what that does is it forces all the analytics to have to go to the back end. So that, to use this example, every time I click on the menu, I send a little packet over to Pizza Hut, and then Pizza Hut can then pass it on to Google, or they could even do the analytics themselves. And so that's where I think we'll start to see more of a need to do these kind of tensor operations and calculating distributions and these other things in Ruby, in Node, in all of these different languages. So rewind your mind back to 2004. What was the piece of software that

started with web that analyzed your Apache logs and basically gave you these really great graphs of like, who is accessing your website from where and all that kind of stuff. Basically, it's fascinating that we're returning to that era of things. I know exactly what you're talking about. I can see the graph in my head. Yeah. Yeah. I ran that for a while for my research lab. Yeah. Anyway, on the topic of like, you know, what does Ruby use for? I think that one of the things that you see over and over again in the apps, the kinds of programs that Ruby is really good at, is they're largely applications

that are highly dynamic. And by that, I mean that they go through heavy development cycles constantly, right? That is where Ruby and Ruby is supremely, supremely good at and Rails makes it even better, right? In that same way that like, for instance, if you were like, I need to write an app, and I'm never going to change it and needs to be as fast as possible. Ruby ends up being kind of a weird use case, because it's like, it's not really optimized for that kind of use usage. Right. And then on the flip side, right. If you write, if you want to write a web, little web thing, and it's going to go inside to say, uh, a wifi router, right. You might write that and see, because I got to just, I got to write it one time and I needs to run in like the tiniest memory footprint possible because it's running on a little teeny MIPS chip, right? But I'm never going to change it. So it might be awful to write the

first time, but I'm never going to go back to it. And so I don't really care, right? And so that spectrum of dynamism is really where you find Ruby at that far end where you're like, hey, look, every day, like Mike can speak to this better than I can. But one of the amazing things about Ruby and Rails is the number of people that you can have that are simultaneously working on the same code base and not stepping on each other's toes. That probably is the biggest feature of Ruby. Rails enables that with convention, but it is astounding. I mean, Mike, maybe you can speak to this. What is the largest team that works on a single app inside Shopify, you think? I mean, the big monolith that we have on any given week probably has 800

contributors. To one app, to one GitHub repo. To one GitHub repository, one Rails application. Yeah. I mean, just hundreds of PRs a day get shipped to production. How do you A-B test that? We test to production. No, I'm joking. So I think we really embrace the fast rollback paradigm, right? It's like we're going to be fast to ship it.

And we have great observability into what's going on in production. And if something's wrong, we can roll back instantly. And so that makes the risk of failure really, really low, which means that we generally will roll forward. If something is wrong at production, we'll roll forward as often as we can. When we need to, we can back out. And we do have A-B testing frameworks, don't get me wrong. But for the most part, I think we're all developing on main. We're pushing from main. And it's managed to scale. We just have A-B testing frameworks, don't get me wrong. But for the most part, I think we're all developing on main. We're pushing from main and it's managed to scale. We just have great tooling.

Wow, that's awesome. Yeah, that is such a, like, if I were to, someone said, you got to spend money to buy a banner that basically says like why someone should use Rails. It would probably be that. It would probably be like, hey, do you want to work on the same app with 800 of your friends? This is probably what like, Hey, do you want to work on the same app with 800 of your

friends? This is probably what you should use, right? Because I think if you were to try to go around and evaluate that condition amongst different languages and different places, you find all of these places where stuff falls down. Oh, well, everybody has to write to this and then that, right? The fact that that is, i don't know that they necessarily meant that to happen but the the conditions allowed for it to to be the case that makes sense what about uh scalability so so i mean clearly rail scales because look at where it's being used but you know how does that actually work so is rails um is is Rails like sitting on a thread pool or do you spin up a bunch of copies of it on a single server? Like what's the scalability part like?

Yeah, that's all me. So one of the things I did during Rubinius was I wrote a web server for Ruby. And it was specifically to make advantage of the characteristics of Rubinius. One of the characteristics of Rubinius was it had native threads available to it that could run concurrently. And so I was like, okay, I need a web server that can really take advantage of that situation. And I coded it in a certain way that it could take advantage of it, but it used the normal API. That web server was called Puma. And that web server has sort of become the dominant Ruby web server today. And it uses techniques that come from WebRick, which is

the original web server framework that actually built into Ruby itself. Techniques that come from Unicorn, which was a web server that was created in the 2007, 8-esque era that still powers GitHub and kind of combines a bunch of different elements in order to give people all the right knobs that they need to do things. So what generally people do is a combination of three things. They utilize a thread pool in a single process. So it's like, okay, great. I know that my app's going to be blocked on talking to the database a good amount of the time anyway. I'm going to use a thread pool and I'm just going to basically handle requests through a thread pool. Ruby's concurrency model is the similar to Python's and that it's, you're not going to have multiple native threads running concurrently.

They're going to be blocked in IO. They're going to switch control, but in a web app, very rarely matters because a lot of times you're waiting on backend for processing and that kind of thing. Yeah. Let me just like dive into that a little bit for folks at home. So it sounds like there's like a global interpreter lock in Ruby, similar to Python. Yeah. So the way these languages work, imagine you have a, so using like Python or Ruby, you have automatic garbage collection. You have all these features that are built into the language. And so it becomes really difficult if you have to handle concurrency there.

So imagine you have a Python instruction that's adding a reference to an object. And you have another Python instruction that is removing that reference. If they can both run at the same time, you have race conditions. And that's just one of many examples where it becomes extremely difficult. So what most languages do is they say, you can only execute one instruction in a process at a time. Now, what happens is your instruction might say, I'm waiting for a packet to arrive over the network.

And so you want all the other threads to be blocked just because this one instruction hasn't finished yet. So certain instructions can what's called release the lock, where they can say, I'm waiting on this network packet. Until it arrives, someone else can execute instructions because I know that I'm waiting for this packet, and I'm not going to change the reference count on the objects, for example. And so then when this network packet arrives, then that same instruction says, oh, now I'm awake

again. Now I'm going to get the lock. I'm going to do some work with this packet. And then when I'm done with that instruction, now it'll go back to the, to the machine to, to decide what instruction to run next. And so, you know, if you, and so in this way, like, even though you can only execute any one instruction at a time, you can still get a ton of concurrency as long as most of the time you're releasing this lock because you're talking to a database or waiting for a network packet or something like that. Yep, exactly right. So you have these three knobs, right? You have threads, which you can tune up and down basically how many requests you want an individual process to handle at the same time, basically locking and unlocking. And you have workers, which is effectively how many

processes you want, right? It's going to use Unix forking in order to basically say like, I need more of these things. And then all of those things aren't going to share memory so they can run truly concurrent, they can share the same socket so they can handle different things uses the same technique that the Apache web server came up with back in time memoriam, right of like, I'm gonna just have multiple processes listing on the same socket on the Unix side, right? And so basically, what I tell people is, you just sit there, and you just tune those knobs, right? You say like, okay, great, like, I need to, I want to hit a certain request per second, okay, I'm going to turn up the threads, you know, at a certain

point, turning up the threads will actually hurt your request per second, because you'll basically be handling, you'll be accepting more requests than you can actually handle in your throughput for how quickly you can get around to threads. So then you, oh yeah, that was too many. Let me turn it back down. Let me turn the workers up a little bit. Because every time you turn workers up, you increase your memory footprint, right? So you just turn those two knobs and eventually you find a sweet spot that's for your app.

And once you find that sweet spot, then you say like, okay, great. Wherever you're running, your container engine, your VMs, whatever you say, like, okay, great. That's our multiplier, or that's our fixed number of requests per second. Now I want to multiply it. And then you just, you just apply a multiplier of instances. You say like, okay, we got 1000, we need to hit 1000 requests per second. And one of our tuning can hit a hundred requests per second, let's just say. So now I need 10 of those things. Great. Let me just fire up 10 of them and you're done.

You go to lunch. Yeah. Is having multiple processes, how does that compare to running multiple pods in Kubernetes? Right. So you can, in Kubernetes, you can say, you know, I want four pods and you might get all four pods on the same machine. Is that like, what efficiency loss is there between that and just running four processes? So there is an efficiency. It's not as significant, but the reason that it exists to do it natively is to take advantage of a functionality that's built into Unix operating systems, which is what people mostly deploy on,

called copy on write memory. So what you will do is you'll load your app in, right? Let's just say you have a fairly big app, and it's going to load the whole app into memory. It's going to parse it, load all the objects, do all the initialization, and let's say that your footprint is 150 megabytes, right? So you loaded all that stuff in, it was 150 megabytes. So you load it into one process. And then what you do is you fork that process. So basically, you make a clone of it. And what happens when you make a clone of it is that clone has its own memory, it never gonna, it's never gonna touch the parent. But what Unix systems do is they actually don't they delay performing the copy until the latest possible time.

So if I can still read memory and it reads from the parent, and if I write to it, it will copy it. That's the copy on write. And so what happens is that if you do that, what you get is memory efficiency by using that sort of worker model where you can fork because you will get this shared sort of memory pool that comes from the original start and that the memory that's used by the individual worker processes will just be the memory that's required to process each individual request. So you can achieve some memory savings. And again, it kind of depends on the size of your

application, what you're doing a little bit with it, that kind of thing. It might also be worth mentioning here, I'll give a shout out to my colleague, Jean Boussier, who has recently proposed in Ruby to have an API call that says, now we are ready to do that copy and write, to fork a new process to do the copy and write thing so that the Ruby VM is now being optimized for the web context that is running in for Rails, which is really interesting. It's like Rails requirements making its way into Ruby. But the idea is that Ruby does as much upfront as it possibly can. And then you signal, okay, now I'm ready to fork so that as much memory as possible won't be written to afterwards like that that mechanism that evan just explained we're going to defer more and more and more of that writing or try to do it up front if we can yeah that makes sense um yeah i saw this uh this really compelling

um it was a blog or a tutorial where someone was using ZFS, which has copy on write as well. And so what they were able to do is basically take their, I think it was their family photos or something that was really large, like gigabytes and gigabytes, and basically copy it this you say oh there's a cron job that's doing you know uh cp directory directory and this directory is 20 gig and it's doing it every day oh my god this can't possibly be sustainable but but under the hood zfs is is is smart enough to say oh just keep references here and if someone changes a file now actually just transparently make the copy um and so yeah fork does something similar for memory i've gotten burned by fork in the past i don't remember exactly what it was but there was something with uh it had to do with cuda and the gpu um so when you start getting into gpu memory and some of these things you have to to be careful. I know Torch now switched to using Spawn

instead of Fork, which does make like a complete copy, a deep copy of everything. But yeah, if you're not doing crazy GPU stuff, Fork is more than adequate. Yeah, I think that there are so many foot guns around Fork specifically, because one of the other... I love that word, foot gun. Yeah, one of the other specific ones yeah yeah what what are the other specific ones and this is probably what you hit is that the semantics of fork are uh let's see it's 2022 now so the semantics of fork are 50 years old now or so and so there's all these efficiencies that were built into fork that aren't useful anymore but we we still pay for by pay for in the quantity of foot guns that are available to you on a daily basis. And so like the resources, so for instance,

file descriptors are shared between two processes, right? So if you open a file and you fork that other thing still has access to the same file. Yeah's cool that's maybe useful it can be very useful it is so problematic if you don't know that it's happening in certain cases right there's two processes that are advancing the file just the file at the same time and trend and getting parts of it and all kinds of crazy stuff things like semaphores and all kinds of os related handles are copied. They're not copied. They're shared between those two. It's really just memory, for the most part, that is a copy.

Everything else ends up being this share. And I think Spawn lets you say, I want a whole fresh new coat of paint here. Don't share any. I don't want the same cup holders as the old one. I don't want the same gas pedal as the same. I want all new stuff. And so, yeah. Yeah. So this is a perfect time to just remind people, if you don't want to have to learn the hard way about fork versus spawn and spend days on it, like we have, use Ruby on Rails. Like, don't build your own web service, you know, that's just one rabbit hole

you'll end up having to go down. And actually, you know, if you're really interested in that level of detail, that's great. I mean, you can work on making any one of these frameworks a lot better. But chances are, you'll want to at least start by doing something high level and using something like Rails. Let's jump a little bit into, you know, what you all have kind of been up to and how folks can kind of contribute and be a part of this. So I think, you know, if someone is, you know, super enthusiastic about Rails, you know, it sounds like like Shopify and Ruby Central are both kind of places where they can go to and contribute and learn more about. Why don't you maybe talk about what are some of the opportunities out there for folks maybe who are just graduating college or maybe they're still in college?

Sure. Well, I think that the first thing I would say is there are indeed Ruby and Rails jobs out there. They're in demand and they pay well. So number one, if you're interested in doing Ruby, you can always reach out to Shopify or a number of other Ruby shops, which is great. There's my plug for Shopify hiring. What is the state with remote? And is it sort of US?

Is it global? Shopify is hiring globally right now. So we, post-pandemic, moved into a post-office world. We don't have offices anymore. Everyone is fully remote. That was a big step change for Shopify because Shopify had kind of built its identity

around in-person offices with great perks, right? Where people could go after work and go to 3D print something or play ping pong or have a great lunch with their colleagues. And the in-person experience is just not available anymore. But the flip side of that is that we can now hire amazing people anywhere they are in the world. And so teams generally find a few time zones that they are affiliated with,

right? So like maybe a team chooses to be mainly East Coast US, but also work a little bit earlier to handle European colleagues or vice versa. They're on the West Coast and maybe they shift a little bit later to have some PACRIM colleagues in Japan or Australia. And so it's a really, really flexible place to work. And if you're interested at all, please reach out. What about internships? Are there internship opportunities for people who are still students? Yes, absolutely. So we have internships, of course. And I think we also have a really great program called the Dev Degree Internship, which is for folks who are in college and need co-op opportunities. They can actually spend a few years alternating between

a semester at school and a semester working at Shopify on a real team. And I think the conversion rate of that program has been huge. A lot of really talented folks have stuck around and taken full-time jobs afterwards. And that's a great opportunity as well. It's called the Dev Degree Program. Cool. That makes sense. Just out of curiosity, Shopify, I believe it was in Canada. Is that right? Before the remote thing? It is a Canadian company at its heart. Yes. Everyone is super nice, just like all Canadians are. Shopify was founded and the main headquarters has always been in Ottawa. Got it. Ottawa is not exactly a gigantic metropolis.

It's kind of, I would call it like maybe a second tier city. It's not in New York or Chicago or in LA. But Shopify built this reputation as being the best tech employer in Ottawa. And that fueled a lot of its growth for about a decade. Wow, that is wild. So is it kind of like totally remote now? Is it since like the lease is disbanded and there isn't an office? So I think we still have some real estate in a few offices, but we use that for like team off sites.

Essentially, we use the verb burst for it because the teams come together and they like burst in happiness and productivity. But they're mainly used as like co-working sites where teams come together for three or four days to maybe finish a project, shape up style. And there's nobody going into that office every day for work anymore. That is wild. So the things you talked about, the 3D printing and the perks, all of that, what have you work to be missing. And so Shopify has done a bunch of things, I think, to try to address this. We do a lot of social events, so teams get together. My team has a show and tell or a social thing on a Tuesday afternoon. We do coffee on Friday afternoon.

We have a couple of other events that we do. And that at least gives us an opportunity to get together and just chat and be social, which is great. We try to get together as often as we can, like at least a few times a year. Most teams will get together in person at these like off sites. Shopify also has really great perks for at home. Like we, you know, I think new employees will get a budget to build out a home office so that they're comfortable at least if they can't have an office to commit to, they can be comfortable at home. And I think that it's a trade-off though, right? So folks who really... I find a lot of folks who

are right out of college, their social lives become the network of people that they know from work. I know I did this, right? I married my wife who I met at work at my first job, right? I think that it is, it's a choice that everybody has to make for themselves, whether they want to be in person now that we're kind of like, I don't want to say we're past the pandemic, because I don't think that's true. But a lot of places have their offices open again. And I think that's a choice everybody needs to make for themselves. But I've been super happy at Shopify. I really love it. Yeah, that makes sense. So when folks do get together in person, that's got to be a fascinating thing. I mean, I remember I recently went to a new job. I took a

new job about a year ago and were in person. But before that, there were people who I had brought on to the team maybe two years prior. And even two years later, I'd never met them in person. And so, you know, getting together, it's really interesting. You kind of have to say, well, you know, here's my sort of, here's the nonverbal side of me. You know, here's like the part of me, like outside of the rectangle of the Zoom call, right? And then also at the same time, maybe we could do something productive while we're here. Yeah, for my day job, I work for HashiCorp, which we haven't talked about here

because it's not really Ruby related, but we do the same thing. And I was just going to say, whenever we're also fully remote and have been for quite some time, one of the things I always have to tell people is, by the way, I'm 6'4", just so you know, because you can't tell at all. And so the first time people meet me, they're like, oh my God, I had no idea. I was like, I know it's because my head is well proportioned

with my shoulders. So there's no way you could tell on Zoom that I'm this tall. And we make this joke every single time. And yet every single time I meet people in person for the first time, they're like, you weren't kidding. And I was like, no, I was not. So I have the same thing where I have my, I think part of it too, is my camera. I don't use my laptop camera because just the way my desk is set up, I would be like heavily backlit. It would, it would look like I'm in a witness protection program or something. So I got a camera and it just happens to have a pretty wide lens. So my, my head and my body just looks small because there's so much other room. And then people meet me and I'm 6'2". And they're like, oh my God, you're so tall. Yeah, that's wild. shopify.com slash careers or slash jobs, we'll put the right link up in the show notes for folks. So if you're listening at home, go to the show notes. Almost any podcast app has a little notes

section. And so we'll put a link in there. You can just click straight in. Cool. I didn't mean to turn that into a whole employment pitch for Shopify, but I really do love working with Shopify. No, it's cool. I'm a big fan of following Shopify. There was a really interesting blog post that the CEO of Shopify made. I'll have to go back and find it. But it really struck me. It had to do with, so basically, you know, Shopify is sort of has this like content farms where they're creating all of this content, but then the content creators are not really connected to the ad system. The ad system is really the centralized YouTube thing. And so it's almost like, like there are kickbacks, but it's completely up to YouTube versus Shopify is like truly a marketplace that's like sort of building up, as you said, a lot of small businesses, which is really nice to see. Yeah, that's absolutely right.

I think the store owners own their own brand. They have their own domain. Maybe the flip side of that would be Amazon, where if you have something to sell, people are still going to Amazon.com and searching. And then Amazon gets to show competitors up against your thing too. And for us, it really is about putting the merchant front and center, allowing them to build their brand, allowing them to build an audience or customers that come back over and over again. And they own that asset, right? It happens to run on Shopify's Rails app on our hardware, but they actually own that storefront themselves and all the information about their customers they own and not Shopify. Yeah, I happen to be looking for an e-bike for

my wife and I was gone to Aventon.com. They sell e-bikes. And it actually is a Shopify site. And the reason why I started going directly to these companies was something kind of, you know, sometimes something small happens, but ends up kind of leaving a big impact. And for me, I was searching for, I typed in Duracell batteries, and I got Amazon Basics batteries as the first choice. And for some reason, that had a really big impact on me. I was thinking, wow, I'm probably missing out on actually the best product here. So yeah, I think it's an amazing platform. It's really cool stuff.

If you're looking out there for internships, jobs, definitely a good place to go. And there's no physical limitation there. So you could be out in Idaho, you could be anywhere and you should definitely reach out. What about you, Evan? Like are there folks, you have hiring opportunities

either at HashiCorp or through Ruby Central? Yeah, I mean, Ruby Central, I think is really small. I think that there's not a lot of positions. And I resigned as a director there a couple of months ago. So I don't have the latest either. But one of the things that I know that Ruby Central is doing right now is working with the Ruby Shield program with Shopify to take some of the, to basically to use the money, the funds that have been income available to hire people. So like one of the things that we're looking to do is hire people who are interested in Ruby

gems and that sort of thing. And I think that that effort is starting up. And I guess if someone out there listening is like, you know what, I have this really great proposal to improve this, that, and the other thing. Yeah, I'll figure out a link in the show notes to shoot over to that for that. For HashiCorp specifically, yeah, I mean, we've been all remote since the inception. So almost a decade now. The co-founders were remote from each other or have been remote from each other for eight of those decades, eight of those 10 years or so.

So we have HashiCorp.com slash jobs, I believe, working on all kinds of different things. A number of Ruby jobs as well. Cool. Yeah, I've used, I think Terraform and Vault are both Hashicorp projects. Yeah, I didn't realize there was a big Ruby footprint there. Yeah, Terraform Cloud, which is our Terraform app, The SaaS offering is all written in Rails. Wow. Did not know that. That is what, actually, this is a bit curious. If you are going to a website, is there a telltale sign that this was written in Rails? Is there something that kind of gives it away? Like I'm trying to think like maybe it's a developer tools. Not as much anymore. As everybody has moved to RESTful routes, it's harder to tell.

You could probably look at maybe some of the headers to try and figure out like, oh, this looks like the pattern that is injected by this asset management framework that's available in Rails. But that's pretty minor because everybody has sort of copied the same techniques over time so not as much anymore yeah no people stop putting the old powered by php buttons on the bottom of the in the footers you know what do you think i'm all for it yeah yeah i love those i love the powered by uh you know linux powered by php little buttons that people put down there so that was in the days pre like what we're talking about at the top of the show of pre-cookies cookie tracking and

all that kind of thing those those images would go into someone's log so they could find out how many people hit this thing so i could figure out how many people are actually on this website so yep yep very cool actually I just kind of a random question. And then we can we can close it out. But you talked about rest API's. Is there a connection between rails and open API or schwagger? Are these other things? Is there a way to generate that? Is that a is that a thing? Or like, I guess maybe the higher level question is, are people using Rails for API servers? Or is it really just mostly for front-end web servers? 100%.

People are definitely using Rails as both a full-stack website, but also just an API server, like 100%. In fact, I think there's actually an option you can pass to the Rails new command, which is like a dash dash API. And what it'll do is it'll just optimize that installation for like a headless API only version of Rails. Shopify's big monolith is primarily a GraphQL API server for the most part, right? Because we do a lot of React on the front end. And so the React is just hitting GraphQL APIs or REST APIs on the back end. And you find that that's where all the complicated business logic is. Anyway,

it's not like it's not easier just because you're not having to render views, right? But yeah, absolutely. Rails is fantastic as an API server. Cool. That's awesome. Well, I want to give both of you some time to just kind of close it out. You know, if there's there's tons of folks out there, a lot of students out there who are just getting started. And what kind of advice would you give them? Maybe, Mike, I'll start with you. Hmm. Stay curious. Hmm. I guess that would be my advice is every every every, um, I think looking back on a 25 year career, all the really fun things I did were because I noticed something that was like broken or wasn't quite right. And I was like, why is that doing that? And then like you do the deep dive and you learn a little bit about your tools and you learn how things break, which teaches you how

they're supposed to work, which makes you a better engineer. And usually you get like a great cocktail party story out of it at the end of it all too, where you go, you're not going to believe what I found at work today. And just stay curious. And as long as you, as long as you're doing that, I think that's, that's the best advice I can give anyone. Yeah. I'll piggyback on that and push it even more. I've, I've talked about this with, with other people before, but like, I'm not generally a life planner, right? Like I don't plan, I didn't plan out a career. I didn't plan out most of anything. Vacations is basically what I plan out. That's about it. And, but I feel like I've been very lucky and looking back on my luck over time, I feel like it's one of those elements where I mostly just took the thing that

was most interesting that really, I trusted my gut about the thing that was most interesting to just go and say like, I'm going to do that next. That's the thing I'm going to do next, right? I'm interested in it. So I had to have interest. So again, to Mike's point, like be curious, like, you know, like if you think that this sounds interesting, like, just do a little research, look it up, like, like, be try different things, right? I think that you will be amazed at how those things more than more so than even a cocktail party, you know, you'd be amazed at like, oh, I looked at this thing. And now it's perfectly applicable to what I'm doing every day or or in your interview. And you may say, like, do you have an interesting story?

And you tell them the story and be like, that's amazing. I had the same problem like those. The tech community is amazingly small, but amazingly wide in terms of breadth. Right. So, like, if you had some weird little thing that you did, you might find that somebody else had that same thing. And that might be you're in to have a conversation with them about a job. Or if you are really interested in this thing that maybe you can't figure out how to do right now, you might just continue to cultivate that and find out that later on. Now there's that thing sort of appears.

It might feel like luck, but really it's you making your own luck. Yeah, really, really well said. You might even get to the point where you've only done machine learning. You could still talk about fork and spawn and how painful they are. Exactly. Someone who's done Ruby. Yeah. It's amazing. A real treasure and a pleasure having you both on the show. I really appreciate it. Absolutely fascinating interview. Thank you again on the show. I really appreciate it.

Absolutely fascinating interview. Thank you again for your time. Thank you for having us. Oh, my pleasure. Yeah. Cool. And thank you folks out there for listening, supporting us on Patreon. It's getting close to the holiday episode. By the time this goes out, it'll probably be about two months away, maybe six weeks.

But Patrick and I are already planning a pretty awesome holiday episode at the end of the year to kind of round it out. And yeah, thanks again for supporting us on Audible and sending us emails. We really appreciate all of the feedback that we get. Thanks a lot, guys. See you later. Music by Eric Barndollar. Programming Throwdown is distributed under a Creative Commonsons attribution share alike 2.0 license you're free to share copy distribute transmit the work to remix adapt the work but you must provide attribution to patrick and i and share alike in kind