The Changelog: Software Development, Open Source - RubyGems and RubyGems.org (Interview)

Episode Date: May 11, 2011

Wynn sat down with Nick Quaranto at Red Dirt Ruby Conference to talk about Gemcutter, RubyGems.org, and how to get started creating your own Ruby gem.

Welcome to the changelog episode 0.5.9. I'm Adam Stachowiak. And I'm Winn Netherland. This is the changelog. We cover what's fresh and new in open source. If you found us on iTunes, we're also on the web at thechangelog.com. We're also up on GitHub. Head to github.com slash explore. You'll find some trending reposts, some feature reposts from our blog, as well as the audio podcast you're listening to. If you're on Twitter, follow Change Log Show and me, Adam Stack. And I'm Penguin, P-E-N-G-W-I-N-N. And this episode is sponsored by GitHub Jobs. Head to thechangelog.com slash jobs to get started. If you'd like to feature your job on this show,

select advertise on the changelog when you post your job and we'll take care of the rest. Asana is looking for a software engineer in San Francisco, California. Great perks on this one, in-house yoga, executive life coaching, organic home-cooked meals twice a day, and the kicker, three 30-inch monitors. Actually, they'll let you spend up to $10K on your own rig however you think best. Be sure and check this one out. Asana

is lg.gd slash aj. Next up is CrowdTap. CrowdTap is looking for a rail software engineer. They're an exciting NYC startup based in a Rails software engineer. They're an exciting NYC startup based in the Union Square area. They're looking for Ruby and Ruby on Rails engineers to join their team. If you're using

jQuery, Rails 3, MongoDB, Redis, the as Wynn says, the usual suspects, Rescue, RSpec, Cucumber, the list goes on. If you're using any of those fun tools, they want to talk to you. And if you want to work with them, check out lg.gd

slash am. Secure Endpoints is looking for software engineers in New York or elsewhere full-time. They develop single sign-on identity management and secure data access solutions, including Network Identity Manager, Kerberos, Microsoft Windows Platform.

So they're also looking for folks on Windows, Mac OS, and Linux, mobile client development, iOS, and Android. If you're interested, be sure and check the short code, lg.gd.ak. A fun episode this week, an excerpt from our live show at Red Dirt Ruby Company. We talked to Nick Caranto from Gem Cutters, now our Gem Cutter, now RubyGems, and the whole backstory of how that project came about and how it morphed into, I guess, RubyGems 2.0 and the philosophy behind what goes into a good RubyGem,

gem spec, and intro to creating a RubyGem if you are new to the process. You've got about a zillion gems out there between all the APIs you work with. I've got several. You've got about a zillion gems out there between all the APIs you work with. I've got several. You've got a couple. Just a couple. It's not as hard as you would think, right? I don't think so, no. I think if you're intimidated, it's your own fault.

You should just give it a try. So easy and Adam can do it. There you go. Fun episode. Should we get to it?'s do it all right we're joined by nick caronto yes from boston yes originally no i'm from buffalo new york known on the interwebs as q rush crush i'm not sure how to pronounce that. Part deux? Basically, that's one way to put it. Talk us through the background of GemCutter and how, I guess, it progressed without stealing too much thunder from your talk tomorrow.

Sure. So GemCutter started as a little side project of mine at Boston Ruby Group. I had just been getting into gems and publishing them. I got in around the GitHub area where you had to check a box and you hoped it worked. And it didn't work and then you checked and someone even wrote a site that you went to it and you looked at it anyone remember this it was like no I haven't seen something so it was pretty terrible

that was pretty bad and I actually tried to sign up for a RubyForge account and I signed up for like a fake, and then they said no. I basically typed in a bunch of garbage, and they actually had someone going in and saying, oh yeah, you can publish a gem now. So I thought that was really not good. So I started talking to Josh Nichols, who wrote the Jula gem, who was in Boston at the time,

and Tom Preston Werner about, what can we do to make this better? And we just went from there. So basically laid out a few ideas about what the site could look like, what it would provide, and just try to figure it out. For a while it was, come listen to Nick's crazy idea of how to kill RubyForge. And then realized that wasn't the best marketing term for it. But I think it's worked out pretty well.

Did you have that plan going in to replace Ruby Forge? I mean, kind of. The way we were looking at it, it was like, there's no other way. So it was either this other weird gem source kind of hanging out that people would be like, well, do I use GemCutter or RubyForge or GitHub? Which one do I use? So I think the plan was just to improve what we had because it obviously wasn't ideal. So I think that's besides the prior motivation I just said, which wasn't very nice. We're nice here. So that was really the driving force was to,

it's going to have to be, and there's no reason for it not to be. Especially with the guys that run the RubyGems project, I feel that you have to really prove that the code you write works. So I had to set out going to that first. Like if I was to go to them and say, oh, here's this idea, they would have been like,

no, that's not going to work. So I had to prove it first, and then I went to them. And that worked. So how did that conversation go about? Did you approach those guys? Did they approach you? It got to a point when Peter Cooper wrote a blog post on Ruby Inside about it.

And this is just after I had gotten it actually working, and people could actually download gems from it. And this is just after I had gotten it actually working and people could actually download Gems from it. And you'd be like, oh, here's this new Gem hosting service taking on these other sites. And that wasn't really the point. We just wanted to fix what we had. So it was around then where I really started realizing, OK, we need to figure this out.

We need to figure out what's going to go forward. So I drafted up a little plan and showed it to them. But before that, I had not really talked to them at all. But luckily, they've been really cool. They've been very nice and open to the ideas we had. And they jumped on it. I wouldn't say immediately, but very quickly. Who here has written and released a gem up on RubyGems?

Quite a few folks. The rest of you, why not? So for the uninitiated, what is a gem? Okay, so a gem is basically a bunch of Ruby code that you can share. That's like the simplest thing. It's a way to share Ruby code. The actual internals of it, it's actually a tarball

that has a YAML metadata, hunk, and then your files. So that's what RubyGems handles, sharing that and tossing it around your system and making sure it's in the right place. And then it also handles actually requiring all that Ruby code that's in the gem at that time. So there's a lot of magic it kind of has

and that's to make your life a lot easier. So I hate to use the word manifest, but I will, but I guess the gem spec is sort of the manifest of the package, what all goes in that? Yeah, so the gem spec has everything from the name of the package, the version, to the date, to the files that's in it, to the description, to an email. If you depend

on other gems, if you depend on other software packages, it's a huge sprawling list of things that not everyone fills out, which is very challenging. Fills out well. You know, an intriguing part about the gem spec is it's actually, you can execute Ruby in it, right? Yes and no. Shouldn't, maybe? So the gem spec, I mean, the spec itself is in Ruby, but it gets saved as YAML.

And that's eventually what it boils down to. You can put Ruby code in there, but when you're packaging up a gem, it's going to save the YAML. It's not going to save the Ruby. Because we can't just arbitrarily execute Ruby code on servers. So that doesn't work out too well, as time has shown. Early in my gem publishing days, I would go back and forth. Somebody would fork the project, submit a patch, but they would mess with the gem spec.

And there was this big hubbub around, do you put the GIMP spec in Git or do you not? If you want to be able to use Bundler from GitHub, it's required, but what is the etiquette, I guess, around the GIMP spec? DAN GALPIN- There's so many different ways to do it. I wish almost Git had a thing that you could say, don't touch these ever.

And Aaron even mentioned it in his talk, to not touch build systems. The way that we tend to do it now is we actually put the gem spec in the rake task and the rake file. So there's a rake gem package task. I don't know the exact name of it. But rake provides a way to package gems and generate the gem file.

And then I'll actually ignore it from Git. So I won't even start in Git at all. And the actual version will be inside the Ruby gem somewhere. And all the information I need will be in the Rake file. That's the way we're doing it now. I don't know what the best way to do it is. I think as long as it's in version control, that's good enough. It seems like Bundler has advanced the landscape of GIM dependencies, or I guess

Ruby library dependencies, to put it another way. Are we still advancing that cause? Are there problems to be solved, or is this the future? As in just managing dependencies? Managing dependencies and basically versionings of those dependencies and things of that sort. Gemsats, the whole nine yards. I mean, I'm sure I'm not the only one who's been waiting on fetching source index, right? Does anyone else hate that error message? I hate it too. So there's a lot of problems to be solved there.

And actually the new bundler release, 1.1, is using a new API that we wrote in Gemcutter to make dependency resolution a lot faster. And that's perfect. That's the exact reason why GemCutter is there. So we can actually, in Ruby, write new APIs that will help out the community and get them out there faster. So hopefully that will be released soon.

We had the endpoint done a while ago, but Bundler is a big project, and it's very complicated. It's not easy to mess around with it, but it's complicated and it does a lot of things. So the war is definitely not over by any means. Who's driving the roadmap of Gem Cutter? Is it totally community driven or do you have a vision for it?

I guess it's, I wouldn't say there is a roadmap. There should be. I should work on that. I would say it's more community driven. We do have a lot of features and feature requests. I try to get them in as soon as I can. But if I'm not happy with the code quality, I'm not going to bring it in.

Luckily, though, there's been a lot of contributors. And I want to make it as easy as possible to contribute. I think the big things we need, and this is kind of something I'm going to talk about. But we need this bigger overarching things we need, and this is kind of killing my, something I'm going to talk about, but we need, like, this bigger overarching things we need that is not really part of GemCutter, but is part of the RubyGems ecosystem. So things like redundancy and mirroring that these, every other open source community has. Like, we get laughed at because we don't have

mirrors for RubyGems. I get laughed at. So there's these problems we need to fix, and it may or may not be within the scope of GemCutter, but the nice thing is that we can actually adapt it a lot faster now. It's not a huge, monolithic PHP site that we are kind of worried to touch. So I guess the takeaway is if you do have mirrors, you have to host them on multiple cloud providers for days like today.

Yeah, I think I know who to talk to. So I guess the takeaway is if you do have mirrors, you have to host them on multiple cloud providers for days like today. Yeah, I think I know who to talk to. So when you go to create a new gem, to the extent that you do, so your own personal preference of creating that gem spec, are you close to the metal manual, or do you like helper tools? Yeah, I mean, I started using Jeweler,

so I tend to default to that. It's gotten a bad rap lately. I don't know, there's so many, I know a lot of people that use Ho, I know a lot of people that hand roll stuff. I tend to go to Jeweler by default just because it generates all the junk and you usually need like a spec directory and features. And it starts complaining at you if you don't write tests,

so hopefully you're doing that. But I mean, I've hand rolled stuff as well. I think the problem with hand rolling is I don't even remember what's going on. The nice thing is that Joomla sets that up and then complains at you until you fix it. And it's not like Ho, where Ho will always complain at you if you don't do certain things and follow their way. So I don't know.

I think just use it the best way that you can and whatever you're familiar with. As long as it's somehow automated, that's the important thing. Who's leading the charge, I guess, in this area? Because it seems like it's kind of a lot of falling forward and you're just listening to the other alpha geeks when they bark at you and do something that probably they figured out a long time ago.

Case in point, around the gem spec in the GitHub or in the Git repository and things like that, but also like Ryan's post, I guess several years ago now, around not requiring Ruby gems. It seems like there's no canonical place to go and say, okay, this is how you do... No, the Ruby gems doc site is not good. And one of the things that was brought up really recently

is that there's no real community place to go like, well here's how we do things. And that may not broadly apply, but at least a general set of rules to say like, okay don't do, don't require RubyGems, don't mess with the load path if you don't need to, don't throw constants in weird places, like at the top of files.

I remember at one point when I was doing the GemCutter gem, which is a gem plug-in. And the gem plug-ins get loaded every time you bring in Ruby gems. And I put a URL constant in there, this rental URL. So anyone that had ever installed that gem had a URL constant defined in their app, always. It's like, I had no idea that was the case.

I just put it there. I had no clue. So I think that's a huge community problem. I don't know who's, no one's leading that. So I think there should be more information on where that is, and that could be a whole separate site, and I'd love to help out with it. What was the transitional gem cutter command that you had to do before the command line

before the Ruby gem switchover? Migrate. That was fun. So that command existed make me think back a little bit. So that command existed when RubyForge was still around because we had to somehow figure out that you owned a gem. And the use case I was always thinking of was that, okay, I need to make sure someone like DHH isn't going to come to my house and kill me

for pushing a new Rails gem or pushing Rails 3.0 by accident. So because there's a few things in the gem spec that we can't trust, like the email. So that command actually would, like, upload a file to your gem's FTP space on RubyForge, which I had no idea existed, and then look for it on the gem cutter side. It was a mess. That's gone now. Thank goodness. But it was a weird transition for a little bit. So GitHub is out of the

gem building business, right? Yes. Is there any, I guess, valid reason to have a namespace gem upon Gem Cutter? No.

Oh, well, yes and no. So, I mean, so the namespace gem, so like your GitHub username and then the name of the gem, those were there because GitHub couldn't do it any other way. And there's been a lot of discussion about how to do forked gems.

And that's basically what it is, if you're forking a gem and you want to publish it. I think now that bundler is around, which it wasn't when we did the cutover, and that you can do a dependency on a Git repository, I think that's perfect. You should just use that. You shouldn't waste time pushing a gem when you can just hook it up right there. And you can even specify a rep.

So if you really want to get nutty with it, you can set a tag in your own repo that isn't going to be in the main repo, and that maintainer will never pull it in. And then make sure that you're always locked to that one. So I think that's a lot better than pushing it up, having to wait for it, and then you have to maintain that gem instead of just the repo.

I would imagine just the sheer virtue of creating gem cutter, you see a lot more inbound gems than the average person. What's the funniest post-install commit message that you've seen in a gem? For instance, the HTTP party. Party. It was party hard.

I'm surprised more people don't abuse that. Because that's the only thing you can do after gem install, is you can actually print the whole message. So I'm surprised people don't like, it's only a string. But I don't know how long that string can get. So I'm surprised people don't like, it's only a string, but I don't know how long that string can get. Right. So. Eric crammed quite a bit of it into the, I'm not sure if this is new and one for the Twitter gem. It's now actually gives you the mailing list and some resources to follow to commit back to the gem, which I thought was a unique way to use it. It seems like the humor aspect of aspect seems to be the more prevalent use case.

I haven't seen too crazy. There's definitely been a lot more silly gems posted, because you don't have to wait a few days, like the meme generator gem and a few gems that do terrible things that I won't talk about. But there's a few funny gems, not so much funny post-installed. But maybe that could be a whole new era of comedy in the

Ruby community. I'll throw the same question at you that I threw at Wes. Gems in a fog library are means to an end. What's your end? What are you building? With GemCutter? With anything.

Sunday afternoon, if you're just coding, if you're lame like me and that's what you do on a Sunday afternoon. Believe me, I'm that lame as well. I'm no different. So I've been working a lot with Redis lately, and I've given a few talks on it. And I'm in the process of writing a service that uses it, so I've just been knee-deep.

This week I've been doing a lot with Event Machine. I've never used Event Machine before seriously for anything. Apparently they do timers really well, which is really hard to write, and they do it really well. So I'm going to let Event Machine do that. So I've been messing around with Redis and Event Machine, trying to wrap my head around it. So besides that, not too much outside of the Jump Cutter world. There's definitely always a lot of pull requests and bugs that suck up a lot of time.

Thanks for joining us. We need to clear off the stage for Dr. Nick's keynote. All right. Appreciate it. Thanks. Thanks, everyone. Thanks, everyone. Thanks, guys. Thanks. Thanks. Thanks.