The Changelog: Software Development, Open Source - RVM and BDSM (Interview)

Episode Date: August 4, 2011

Steve and Wynn caught up with Wayne Seguin to talk about his Ruby enVironment Manager and BDSM shell scripting framework projects.

This week's episode is brought to you by Harvest. Harvest is a web-based time tracking application relied on by creative teams and freelancers in over 100 countries. With Harvest, you can easily track your billable time and invoice for it too. If you're on the go, Harvest has native iPhone and Android companion apps that make it easy to track time and expenses from anywhere in the world. Try Harvest today with a 30-day free trial at getharvest.com and use coupon code TIMESAVER

at checkout to get 50% off your first month billing. Welcome to the Change Log episode 0.6.6. I'm Adam Stachowiak. And I'm Winn Netherland. This is the ChangeLog. We cover what's fresh and new in open source. If you found us on iTunes, we're also on the web at thechangelog.com. We're also up on GitHub. Head to github.com slash explore. You'll find some trending repos, some feature repos from our blog, as well as the audio podcasts. And if you're on Twitter, follow ChangeLogShow and me, Adam Stack. And I'm Penguin, P-E-N-G-W-I-N-N. Fun episode this week, talked to Wayne Seguin from, I guess, Engine Yard fame as of late.

Probably know him from RVM. Definitely using RVM. RVM is the bomb. Also talked about BDSM, not what you're thinking about, but a Ruby scripting environment that's on the show. I kind of asked the question if it was part homebrew, part chef, part RVM, and he answered indirectly, yes. So it's kind of all those things kind of mishmashed together. It's really fun. Is this the same BDSM that Steve posted about two days ago?

It is. One and the same. Steve's played with it a two days ago? It is. One and the same. Steve's played with it a bit, so he did a quick post on the blog last week. But it's fun if you like to have your own scripting environment that's the same on your local machine and all of your unisys as you deploy your codes. Gotcha. A couple of quick programming notes. So we'll be in Austin, Texas for Lone Star RubyConf. We'll be doing our Design Eye for the Dev Guy slash Gal all-day training on Thursday, August 11th. And I'll be giving a TI talk on the 12th. So look us up in Madison, Wisconsin. Our buddy Steve Klabnick is going to be giving a talk up there, contributor to the show, co-host actually on this episode as well.

Also featuring Scott Chacon and some other friends of the show up in Madison. Good stuff. Absolutely. Fun show this week. Should we get to it? Let's do it. Chatting today with Wayne Seguin from RVM fame. Wayne, for those that don't know, why don't you introduce yourself and a little bit about who you are.

My name is Wayne Seguin, and I wrote a small 8,000-line or so shell scripting framework for managing Ruby environments. You can find it at rvm.beginrescuen.com. And in addition to that, I've also wrote a new framework for system level shell scripting called BDSM. And that you can find at bdsm.beginrescuen.com. I also currently work for Engine Yard under the auspicious personality of Dr. Nick, who has been known to wear fairy costumes. We've had Dr. Nick on the show. Lively personality, to say the least. So what was the driver behind RVM? Some pain that you were trying to relieve?

Yes, actually. Basically, I left Engine Yard during my – I had an initial run at Engine Yard. I left Engine Yard, and then I went to work for another company, and they were trying to – what they hired me for was to help with their infrastructure setup and deployments and all that stuff. So I came in there, and what they needed originally was they had three projects. One was on JRuby, one was on 1.8, and then their new project they were starting out was on 1.9. And we had to run all three of those Rubys, and there was really no nice and clean way to be able to bootstrap those three rubies and to be able to manage their applications. They needed a way to be able to have their development environment

and their CI, QA, production, demo, staging, all those environments identical without having to go through too much pain or time. So I wanted to learn more about shell scripting, get better at it. So I basically sat down and taught myself shell scripting, found the best sources for it that I could find, and started learning the do's and the don'ts, all the different techniques and what it can provide, what it is.

And then I took that and I started iterating over what was primordial RVM at the time. And I got an initial Ruby environment manager up and running. Back then I called it Ruby version manager, but that's not quite as an apt description as Ruby Environment Manager. So I started using it, and a week after I started, I wrote it over the initial version over one evening, and then I iterated over it for a week. I had it working pretty decently. It would basically download, compile, install rubies and keep them semi-isolated as a user install. And then I showed it to this guy named Peter Cooper.

Who? Some guy named Peter Cooper. Friend of the show. Kind of goofy, you know. But anyways, I digress. So, yeah, he did a blog post about it and then next thing i know i had like this irc channel and people were asking me left and right to add features to it or actually what they were really asking is hey could i do this and i was gonna be like no hold on a minute

and then i go in five minutes later be like hey uh get ahead and check this out and then it just kind of snowballed and iterated and so was it the move to 1.9 that prompted it or was it jruby and re and some of the other flavors actually it was both 1.9 and jruby the basically they had a a massive legacy not exactly legacy but they had an application they had written and it was using MRI 1.8. And they were embarking on a new application, that is, they were going to write it with Ruby 1.9, and a piece of it was supposed to be an ETL processor and needed to use JDBC to connect to SQL servers at the time. And that was a JRuby piece. And that's where that came in. So it was a new

project that they were starting out, which is now turning into their main business project. And it used Ruby 1.9 and JRuby from the start. One of the big things that always lets me recommend RVM to people is that IRC channel. And from personal experience and just hearing from other people, you're exceptionally good at being available and helping people out. So how do you manage to stay up for 20 hours a day and hang out on IRC and push new features out to RVM so fast? I'm not sure I could answer that question if I tried. For the last couple of weeks, I've kind of been dealing with some family things and personal things. So I haven't been quite as available. However, overall, yeah, for the past two years, I've been overly available. Yeah, I don't think anybody can begrudge you for spending time with your family after this like, you know, I don't know.

Yeah, I think I basically, I don't know. Yeah, I think I basically burned myself out a bit. But much to my happiness, the community has started to step in and help out. And there's a lot of people on the IRC channel now that help out. And even more phenomenal to me is I now have a co-conspirator, Michael Pappies, who is out of Poland, has stepped in and he basically took the ball and started running with it and let me take a mental sanity slash family break. He's been helping support and drive RVM forward for the past three or four weeks now. So now that I'm kind of getting back into it again, I can't iterate it enough that having someone helping on a project that everybody is using like this is just unbelievable. Before that, I had a few quick helps here and there, a few random pull requests and stuff like that, but nobody that actually knew the codebase inside out since last year's Ruby Summer of Code, where I had dedicated Darcy Laycock for that entire stint of the Ruby Summer

of Code. Since then, I haven't really had anybody that stepped up and helped out. And Michael now is stepping up and helping out with both RVM and BDSM. And it's just amazing how much more energetic I am able to get about it now. You know, one of the powerful features of RVM are gem sets. There's a post by Ryan McGeary, I guess is the name, talking about how as a community, we're probably abusing gem sets for application development and we should be using the vendor everything approach. What's your take on gem sets and when they're useful? Well, personally, I use gem sets on every project.

I bundle, I vendor nothing. I tend to, I do firmly believe in complete and utter isolation of an application. What I do is I do use gem sets to do that. And basically I obliterate, if I'm deploying to a server, you can just start clean or obliterate a gem set. And if you're using Bundler or Isolate or something like that, or RVM gem sets themselves have a gem set import feature, an export feature. And you can use any of those things to bootstrap exactly the gems you need for your application. to a system that's insanely locked down behind firewalls such that there is no actual good reason for them to be that locked down,

however they do it anyways. So anytime I deal with people like that, I tend to actually bundle things, but I don't actually bundle them in the application. I like to keep my application's code bases lean and clean. So I have another way that I propagate bundled gem set directories and stuff like that. A big piece of the workflow is how Bundler works together with RVM. How much integration have you done with Yehuda and how he designed Bundler to get those two to play nice? Originally, there was a lot of back back and forth with yuda and i and we got a lot of the

kinks worked out right around bundler 1.0 and shortly there after 1.0 point you know a lot of iterations and they started to play nice together and then sometime recently they did uh things started to diverge a little bit again. So what I've requested – actually, I talked with Indirect about – and apologies. I don't know. I don't remember his actual name. I deal with him mostly on IRC and Twitter. I do that all the time.

So – Yeah. If Steve Kladnick's Twitter handle wasn't Steve Kladnick, I wouldn't know who he was. Yeah. People have trouble remembering my name as well. So basically we went back and forth for the brief discussion and we figured out a way to do it. And actually just yesterday or the day before, Yehuda popped in my channel and asked me about my references to that discussion. And what it is, is what we're going to do is I'm going to have RVM basically say, hey, I just entered a project directory. Is there a gem file? Yes. Okay. It looks like we're using bundlers. So basically what it's

going to do is bundler is going to have its own special binary path the bin directory right i'm going to add that to the beginning of path and the beginning of gem path and that's where um everything will be pre-loaded now the only bundler change that they need to make is to have bundler respect loading of from gem path instead of only respecting gem home like it does now. So in other words, full RubyGems support. So loading can come from anywhere in the gem path, whereas installing only goes to gem home. And also their special bin directory so that they can inject loading of bundler into their binaries

in this special bin directory but not interfere with RubyGems proper. And what that'll accomplish is people will be able to use Bundler and have it respect all of their environment and not have to type BE bundle exec, anything like that, in front of their commands, which is, frankly, it's an abomination. It goes against everything that RVM stands for with cleaning up and keeping a central, unified API for everything.

With RVM, you don't have to type MacRuby, JRuby, IronRuby. You don't have to type all these different binary names. You simply specify which one you want to use, and then you type Ruby, Gem, IRB. It's all the same. I had to go through Flaming Hoops to get it that way, but once it was that way, then everybody else benefits. They have this same workflow no matter which Ruby they're using. I want to come back to the unified thing in a second, but before that, there's sort of two ways that Bundler has these binary stubs and wrappers, right? So we're talking, in this case, this is about the bin stubs feature of Bundler,

not the wrapped up special Ruby bin stubs thing that RVM has, right? And what is that used for? I've never actually seen anybody use that feature. I noticed it was there one time where you can sort of generate a Ruby with its own special name, but I haven't actually seen anyone use that.

So what was the impetus for that? And what's it good for? Well, that's something completely different than the idea of bin stubs with those. But you can actually name your rubies, anything you want. You can alias them, which basically records into an alias file that the – when you alias a Ruby, it basically allows you to, like, say, alias, you know, 19 and then to 192 or patch level 280 or whatever. And then you could say just from then on, anywhere you would put a Ruby string specifier in our VM's command line,

you can literally just type 19 or something short and convenient for you. And then there's also a there's also a wrappers concept where you can generate a wrapper for a Ruby and a command. So using this wrappers feature, you could actually

say, for example, Engine Yard, Heroku, they have their own gems. And it'd be nice to be able to use those gems anywhere in your system without having to worry about switching to the gems that you installed them in to use and stuff like that, right? Yeah. Well, you can use the wrapper feature in order to wrap those CLI tools

and peg them into, and what it does is it pegs the environment to whatever environment they were installed in. So you can actually, since you're using RVM, the RVM bin directory gets into your path, and you can now type Heroku, EY, Heroku deploy or whatever it is, EY deploy, all those things.

And anywhere in your system, no matter what Ruby or gem set you're using and it works as expected yeah that's that's awesome um the unified approach thing definitely helps when getting people acquainted with you know or just in general not having to remember how to do things 12 different ways is really nice but sometimes uh you know things since you're sort of the gateway into ruby you have to know a lot about a lot of things like you said you sort of went through a ton of effort to make this happen and sometimes there's a little bit of you know back and forth like what just happened over the

last couple days with the nail gun situation that i talked to you about so uh michael if everybody doesn't who isn't obsessively using jruby and paying attention to RVMs, Michael added a thing that starts JRuby with Nailgun. So when you do that, type Ruby to start JRuby, you get Nailgun, and Nailgun can change the way your application works. Usually it's faster, but sometimes it breaks, and sometimes it's slower. So how do you keep on top of 12 different versions of Ruby

and their patch levels? Do you sort of take more of a pull approach and assume that people will just, you know, say, Hey, Wayne, this is broken. Or do you actively pay attention to all these different projects? How does that sort of work out? So I do not actively pay attention to all the projects because when I change something in RVM, I will know within five minutes. No, I'm serious. I guarantee I will. I will know within five minutes whether it broke anything. That's awesome and terrible. Well, it is.

So basically anytime I make a change, I literally make sure I'm just waiting around for the next half an hour. And if anything broke for anybody, I can fix it right away because I know exactly what it was. And, yeah, it's so easy to change things that way because I get feedback. And that feedback is absolutely priceless. It's like having an army of QA. I checked it out. And the RVM website and the BDSM website, between the two of them, they're serving just for the website documentation and everything like that. It's serving well over, I think it was like 2 million requests a month.

That's amazing. And then you add into that all the downloads and stuff like that. It's like, wow. That's awesome. But yeah, so what happens is I change. Usually I don't really just, I don't anymore change things just to change them. So I will change things to refactor to clean up the code base. However, as far as like adding new features, I really at this point only do that if there's a user who says, hey, does it do this?

Can it do this? And if it sounds like a good suggestion, sounds like a useful feature, has a use case, I will just add it and push it and then have them test it. They test it. It'll be good or bad. And I'll fix it real quick. And then once they've got it and tested working, I'll just wait and see if anybody else screams or squeals. And if it all looks good, then I'll push a release out either that same day or the next day or whatever switch gears for a minute talk to us about bdsm yeah bdsm is uh originally it was the bash deployment and server manager and because it was that's actually what i built it for originally

was to manage all all of my uh various i was managing like 82 or more servers for different people and I wanted something that would keep them all the same. They were among many different operating systems ranging in age from anywhere from like, oh my god, why is this thing still chugging along to, woohoo, it's Arch Linux, the latest and greatest. That kind of stuff. So, yeah, so originally it was meant for setting up

and deploying applications on servers and managing that. And then I started thinking and dealing with more, as a systems administrator, I had literally an accumulation of random scripts and things that just kind of started collecting on all the systems and and i would go and i'd need one i need a script and i'm like oh crap where is which system okay okay oh this this friend of mine had it for his systems, and okay, I'm going to go to one of his servers. All right, now where did I store that script?

Let's do which. Oh, no, it's not there. Crap, I know it's on here. So, you know, trying to figure out where it is, and not only that, but then you also have the issue of revision control and all this other stuff with these scripts, right? So it can be getting a little bit painful for that. And here I had this great little system this bdsm

and i had this fledgling modules concept in it and now this was like a year and a half or so ago i started adding into it the concept of modules and loadable modules and extensions. And so let me briefly give you a rundown of what those really are. So it changed from this server setup and deployment script into a full-fledged system-level scripting framework. You can actually now put BDSM in the shebang line and start using its modules and extensions inside of your scripts, which gives you nice things like stack traces that print out such that based on your editor setting, as long as your terminal supports it, you can command click on the line and it opens up to that source line and file and lots of other stack tracing and application tracing and debugging goodies as well as a lot of DSL constructs. Instead of having to remember the bracket, bracket, dash, S, file name, and, and, this kind of stuff, you can literally do if space file underscore exists and then the file name or the path to the file with the file name. If space file is executable, you know, there's all kinds of DSL methods that are provided by the module, the core modules and the file system modules now by default. And using these modules, so the are contained basically namespace sets of functions

for doing different things. Like there's a file system, a system, a user, a blogging trace, uh, all kinds of different modules. And so they're basically like equivalent to in Ruby, the standard library so that they're along the same idea as like the Ruby standard library, but for like shell scripting. Yeah, like active support is sort of almost what it sounds like. Just like these are all useful things that you might like to use. Something like that. Yeah.

And it does two things. It extends the features of shell scripting so that you get very clean looking shell scripts and also makes them, gives them all these enhanced features like, yeah, sure, the active support concept. And also there's a lot of extra error checking. So if you use the DSL functions, literally any and every error situation that I could think of when I'm writing those functions, I account for in them. And if there is an error scenario in them, I have it spit out a backtrace as well as the exact message. So if it's clear that it's a coding error, like you say, if file exists, then blah, blah, blah. Well, clearly you didn't specify the file. Well, it'll actually spit back a stack trace showing you how it got to there.

So you can see what line of your code was calling if file exists. And it'll say, error, you didn't specify the file name to the function or something like that, right? So it gives you a lot of sanity checks and helpful stuff when you're scripting in shell. Now, using that as a basis, the BDSM extensions are basically a whole other level of nicety. Extensions are to be thought of as namespaced sets of actions. And actions are basically either shell functions or scripts. So if you have scripts to manage your, let's say, Redis, for example, you probably have a script to install Redis on a server, a script to start it, stop it, maybe check the status of it, that kind of stuff.

So what you would do is you would actually write a Redis extension. And in the Redis extension, basically you have an actions directory, and inside of that actions directory you can have as many nested subdirectors as you want. And in each one of them you can either put executable script files or a.actions file which lists a command line action like, say, BDSM Redis package install. So I'll have a Redis. The Redis extension will be a directory, Redis slash actions slash package slash, say, I do the executable script way. It'll be the script there called install.

And then BDSM, once the extension is installed, as long as it's executable in that directory, it will just call that script file, the install. And that install script can be any language as long as it's an executable script file. So you can write it in compiled C. You can write it in Python, Perl, Ruby, Shell. Now, BDSM, if you do write your stuff in Shell, then you have the edit bonus of all of the standard modules that BDSM provides. However, you're not restricted to do that.

And so then what it does is BDSM encapsulates your sets of scripts, your namespace sets of scripts in extensions. So you can have one for, like, say, Redis. You can have one for deploying, which these are actually examples that I have. One for, say, Unicorn to control it. And then you install BDSM, you just say BDSM and then the extension name package install for, like, Redis. So BDSM Redis package install installs Redis. BDSM Redis service start, stop, restart, that kind of stuff does start, stop, restart on Redis. And those are implemented as system-level shell scripts or Ruby scripts or whatever you want. And one of the things, if you go to RVM's website, you have slash deployment slash best dash practices.

And I need to update that with a new API. But essentially that details how you would go about deploying, say, Redmine as an example application. And the idea is that you can use BDSM to bootstrap your application stack and control everything application-related on the system, and then separate from, like, package manager and everything like that. So what this affords me is, if you recall, with RVM, we have this isolated system where we can specify the environment, which is like the Ruby and the gem set and the list of gems and that kind of stuff. And so for your application, that becomes the application's environment.

Well, this is taking that one step further to the entire application's environment and system. So given any Linux-type operating system or BSD, including OSX, you can install BDSM as root and then start managing your application stacks with BDSM, as well as all of your writing extensions for your sets of scripts and stuff like that. So there's just so many things that you can do with it. It's ridiculous. So this is sort of part Chef,, chef part homebrew part.

Oh my Z shell for bash. Uh, yeah. So it's not just bash though. It's, it's more of a system level framework. Um, it happens to be written in bash at the moment, but, uh, so the idea, the fundamental idea that you can think of is in the reason why I made it the way I did is, so I was trying to extend the RVM concept to my entire application stack. Since I had so many different servers that I was helping people manage, then they were on all kinds of different operating systems while trying to account for differences in package managers and package versions and names on them and everything just became a living hellish nightmare for me. So instead of doing that, what I do is we have some base libraries that, you know, we have the system, we have the system and the systems package manager install and manage the system itself.

And then each application between BDSM and RVM, we manage the entire application stack specifically using those two tools. And it's cross-system the same. It's also completely isolated inside of itself, just like RVM is. Where are you installing these two? What sort of paths? So for BDSM, you can do user or root installs. I only have one person doing user installs, so it's not heavily tested. It's an enterprise-type shop where they have the full barrier between sysadmins and developers, and developers only can get in as users and stuff like that.

So they are doing that. So it does work that way. But the way I have it done on all other systems is it's installed as root, BDSM is, and then every application gets its own system user and its own RVM install. The only user thing tripped me up the first time I ever used BDSM. Now, I have a server that I've been deploying two applications to with it, and it's been awesome, but one of them is called Deer as far as bdsm goes because i didn't realize that that was the way that it worked and i really like the way that it

works uh but just a hilarious i'd always i've been meaning to move it over to a real name and i haven't gotten around to it yet because it confuses me every time i try to configure that particular application yeah it works well it's awesome um it's it's so much simpler than Capistrano was to manage everything. One of them is a Sinatra app and one's a Rails app, and one's using MongoDB and one's using Postgres, and so all that kind of stuff is way easier to manage so far. So I'll be reading that best practices page. That's exactly the plan.

So for application developers, like web application developers or people like that, the idea is that you need to manage your application stack specifically. And you want to be able, in my opinion, at least the way I do things, is I want to be able to go on my laptop, on my desktop, on my CI server, my staging server, my production server, all these, right, the QA, and I want to have them identical. I don't really give a woo-hoo whether it's on OSX or this one's on CentOS or this one's on Arch Linux or this one's on RedShit.

I mean, basically, I really don't want to care. I freak, I really don't want to care. I want it to work the same on all of them. And also if it's set up and run and works the same on all of them, that also enables me to kind of test exactly that application stack as I go from development to QA to CI and staging, right, all the way to production. If it's identical, the entire application stack the whole way, then you're more confident that it's going to work the same as you bring it on. You run into any problems supporting cross-platform scripts like this? Okay, so basically supporting cross-platform scripts is a living nightmare, but the nice thing is if you use the BDSM underlying core DSL functions to write everything, then those I've been ensuring when I write them that they're cross-platform as far as cross, like, you know, star next distributions. And so as long as you're using them when you're scripting, then you automatically have these extensions and scripts that work correctly, no matter whether you're deploying to Fedora or, you know, CentOS or Debian or any of those things.

Not sure if I answered the question. Oh, absolutely, yeah. It just seems like a tall order. You know, Homebrew is OS X only, right? Yes. So stretching across any Star Nix platform just seems infinitely more hard. Oh, it is. And I'm just tackling it one piece at a time.

As I find and need new... So there's two big – so there's like core modules. There's like base level modules like file system and user and they're basically dealing with one single small concept. But then there's also complex or compound modules and two examples of that are package and service. So the package module encapsulates the idea of packaging, right? And the service module encapsulates the idea of services, things you run on a system, start, stop, restart, status, that kind of stuff. And as I need new things for applications, like, okay, I need Redis, I need Elasticsearch, I need Postgres, whatever, I'll sit down and I'll write an extension and I'll test it out,

make sure that the extension is using the core modules and package module and service module so that I get all the maximum benefits of a generalized framework for package managing, for service managing. And in doing that, you've got a common command line interface for all these different services, right? Actually, that is exactly correct. That's another thing that I'm not stressing enough is also it's a common command line interface to it. So yes, so if you're using the service module, all your commands can be managed based on like BDSM, your extension name,

service, start, stop, restart, status, et cetera. Similarly for package, install, uninstall, update, whatever. And what's nice about this is it's a common command line interface to – and you can make scripts that do this or what's even neater is BDSM is a single systems scripting framework, really. That's its focus. I'm not trying to conquer anything broader than a single system scope with this framework. So the idea, however, is with this framework and its extensions, you are able to provide a very succinct common command line interface to your extensions to be able to control and update and install your entire system. And then you can now use distributed tools such as Puppet, Chef, whatever you're going to use, SSH in parallel from a shell script, to call out to all your servers and services

and do all the logic of managing what should be done on each one of them. And they will say, okay, BDSM, blah, blah, blah, BDSM, blah, blah, blah, BDSM, blah, blah, blah. And the idea is that you're isolating so that BDSM worries and manages specific aspects of the single systems. And then your chef or puppet, whatever you're using, will manage your overall infrastructure and coordinate it. And then also all of your recipes and stuff like that for Chef become much simpler because you're literally calling out to the BDSM command line interface. That's one issue I've always had with Chef and Puppet is trying to manage system level things in Ruby and in these scripts has been, for me, a nightmare as it's just so unnatural.

It just doesn't make sense to me. Whereas in shell scripting, what you're doing is you're just stringing together commands with a little bit of logic around them and piping their inputs and outputs together to manage your system. with a little bit of logic around them and piping them to their inputs and outputs together to make, you know, manage your system. And every single binary on your system is a command available in natively inside of your shell script.

It just makes a lot more sense to be doing system level management inside of shell scripts. Now, another thing that I'm trying to do with BDSM is through these DSLs and everything. I'm trying to clean up shell scripting, show that it can be very clean, readable, and debuggable.

Whereas, if you look, just go SSH into your favorite Linux distro and look through the ETC directory of the shell scripts in there, and basically it'll make you want to claw your eyes out and... Do you provide man support for these? For which? For any of your scripts, do you have like a for these? For which?

For any of your scripts. Do you have like a default documentation, man files or anything? There's a BDSM help feature for now. However, we have currently planned providing like hooks into man pages for BDSM extensions and everything itself right now. Very cool. You were talking about being able to stitch these together. When I got into web development, it was back in the day, ASP Classic on the Windows platform. I couldn't do

anything that didn't involve a right click property pane somewhere. Having gotten into Ruby and Rails and the command line interface, I just could never go back to something that I couldn't just stitch together something that I wanted to do in the script, you know? Yes, I do know. That's exactly it.

It's insanely powerful and flexible and... nerd on. So, yeah, it's... Go ahead. I was just going to say, so there's actually sort of two related things about setting all this up. So, obviously, if you want to have one system control your entire application stack

and all these things, installing BDSM, I mean, I did it, so I sort of know. But if you're going to talk about how relatively easy it is to actually get started and how much work do you actually have to do to get a new blank, you know, I spin up a new VPS, how long is it until I'm deploying my code? And then a sort of little follow-up question about the whole getPipeToSH install method that people have been complaining about a little bit. So first thing, how quickly is it to get a new VPS actually set up with all this stuff to get going? Depends on the size and the computation power of the VPS.

I have been focusing on the size and the computation power of the VPS. I have been focusing on the compile and install for the specific system. This allows me to hit a broader range of systems out of the gate without having to worry about whether or not what I built would be compatible. And if the dependent libraries or the dependency libraries like, you know, Nginx, okay, well, that requires PCRE and Zlib and OpenSSL and, oh, I'm using this extra patch to it, so I need this other library on my system. So all of the extensions that I've written so far take on a compiling by default approach. Now, you can write extensions that don't do that, and there actually is one extension that doesn't do that.

For example, I think it's MongoDB, if I remember correctly. It's basically download the XCubital for your platform, extract, and copy the files into the proper locations, and you're done. So, you know, it's possible to do anything you want with it. Now, that said, on an RVM test server that I have, it's not a particularly beefy server, but it's got, you know, like 2 gigs of RAM, and I don't know how much processing power, but within, I don't know, 10 or 15 minutes, I have a Redmine instance deployed. And most of that time is literally just waiting for the compiling on it. So if you use Gentoo, you'll be right at home as an Arch person not to crack jokes at Gentoo all the time. I'm actually a fan of both Arch and Gentoo. So similar. It's like Ruby fan of both Arch and Gentoo. So similar.

It's like Ruby Python is what it reminds me of. They're like almost identical and there's no reason the two should hate each other but they seem like they do. It's even more fundamental than that. It's like Rubinius and JRuby. So to the second piece I guess of that poorly worded

question from earlier. So RVM definitely has the ability to install through like curl this shell script and pipe it to SH. POW does this. NPM does this. And I see complaints on the Twitters all the time about how this is a terrible idea for installing software. Yeah, let me put it to you this way. It's a shell script. Your shells are capable of reading streams of files, whether it be from curling, whether it be from catting, whether it be from reading a file, and executing those files in there. So this is literally quite equivalent to downloading the file, making it executable, and doing .slash install. There is no difference. Now, there was a possibility for a man-in-the-middle attack, which is why I bought the SSL cert for RVM, thanks to the donations. And so then I set up the SSL and then had lots of lovely little nightmares with CA certificates not being updated on people's systems.

Whee! Hold on. Tapping my thing. Hold on. So, yeah, but – so that preempts the man in the middle thing. But if they're still going to complain, then they're just quite literally being pompous jackasses. And the reason for that is you shouldn't be running anything on your system that you don't trust for any, any number of reasons.

And so if you have a problem with that, download the script, read the code, see exactly what it does. Oh my God. What a concept. Whoa. You just blew my mind.

So anyways, I don't know what else to say about that. It's like, go ahead, complain. You know, it's like you're just showing how much of an ass you are, really. So Wayne, how long does it take you to bootstrap your own personal setup? So Lion comes out, you want to install from scratch. How much of your own setup do you have automated uh i'm a very bad monkey as far as that's concerned i rsync over my project directory and then i basically just do the uh curl and cat and pipe and smoke it. RBM and BDSM installs, and then I just take it from there. I basically will install the rubies as I can.

I'll literally open up like 10 shells and 10 different terminals and install as many as I can in parallel. Usually I have to install like one eight or something first and then do all of the rest. And for BDSM, I have that down to a little bit more of a science. I could actually, at this point in time, I could script the whole thing. Just open up a file, make it shebang line, user local bin BDSM, and then just start calling commands in there

and or just have a normal shell script where you do BDSM and then just start calling commands in there and or just have a normal shell script where you do BDSM NGINX package install, BDSM Redis package install. Or there's actually a new thing that I just introduced, which is an alternative syntax to that. And in literally one command, you can have it install all of your package things. It's kind of fun to use, too. But basically, you can do like, you know, for example, BDSM zlib, pcre, openssl.enginex, redis, space package install. And what that will actually do is that will, in parallel, at the same time, install, what did I say, Zlib, PCRE, and OpenSSL.

And then as soon as all three of those finish, then it will launch an install of NGINX and Redis in parallel. Ah. And the reason for doing that, well, hopefully it's obvious, is because before you can build NGINX, you need those dependency libraries to be built. So I've got this compact command line syntax that I introduced today that allows you to quickly in one line install a dependency tree.

And it also respects the number of CPUs on your system. So if you have two CPUs, it will only... So in my example, let's say you have two cores or two CPUs, using my example of PCRE, Zlib, and OpenSSL as the first set in sequence, it will install Zlib and PCRE, and it won't continue to OpenSSL until one of the two of those finish. That's awesome. And then once all three in the set of parallels finish, then it will install a, you know, Nginx and OpenSSL. And so, like, on the RVM's main test workbench that Engineer donated, it's a 24-core Mac Pro.

And so on that one, I can make a massive string and watch everything light up, and it grabs some popcorn, and it's awesome. I mean, seriously, it gives me the biggest nerd on ever, which keeps me building more and more packages so I can see it. Anyways. Another RVM question for you. So any way to have it automatically create a gem set that it hasn't seen before when I click on it. In the documentation, if you actually read it, or in the example slash RVMRC, there is an RVM gem set create on user, something like that. There's actually a flag for it. Dash dash create is the flag.

Okay, so from the command line, you can say dash dash create, and it will create it. Or you can make it so that any time it sees a gem set, it doesn't exist yet, it will automatically create it. That you can put in your.rvmrc file in your user home directory or your etc.rvmrc file. And you're going to have to look in the documentation or in examples slash rvmrc. But it's something like rvm gem set create flag or something like that.

Awesome. That's pretty much the first thing I do on every project is make an RVMRC with a gem set as the name of the project and shove it in Git. I actually do that as well. rvm-rc-create 1.9.2 at my project name. And that will generate the.rvmrc. I do git init, git add.rvmrc, and then I continue. So while we have you here, I know this is in the RVM Best Practices website, but let's get it straight from the horse's mouth. RVM RCs, always check them into your repositories and have a project gem set with the same name as your project, right?

Yeah, completely correct. If you're, if you, the RVM RC is the bond is the specification of this is the project's environment. When you're in this project, when you're running this project, it should be in this environment by default. So that when you go into the project, whether it's five months down the road, whether whatever, whether you, you know, oh, crap, I lost my system,

and I just cloned the repo CD into it, RVM sets up your environment, and everything's kosher, and you get going. And all additionally, keep in mind, the RVMRC file is actually a shell script. So it's not just

setting the RVM Ruby string and environment ID, which is the Ruby string at gemset name. It's more than that. You should do in the RVMRC with proper error handling. Anything that you need to do to set up this application's environment to be ready to work on it or run it.

You know, when I see anybody complaining about RVM or Bundler on Twitter, I just want to remind them where we've come since 2005. Gone are the days where it was just like a picture puzzle game of dependencies trying to get a Rails project to run back then. Yeah. Thanks for that. Thanks for those memories. So basically, it's basically a contract between all developers on that application. And since it's a shell script, you can actually put checks in there that will do things like check the git branch. And based on the git branch, so if I'm on production, then we're using this Ruby.

But for the master branch or development branch, then you would use this environment identifier instead because we're on 1.8. That's production. But if the Git branch is development or master, then we want to be on 1.9 because we're working on upgrading on the master branch or something like that. That is the proper way. That's awesome. Or if you've got a special Ruby for Heroku, you know, that's different from your local too. That's cool. Yes, that is the proper way to be using your RVMRCs. And if you're a Bundler user, then if you look in the generated RVMRC file towards the bottom, what you will notice is that it has examples of what I... Okay, so let me back up. It has examples on there. What I actually do is if I'm using Bundler on a

project, I will have the top of the... the very top selects the environment ID. The middle is what is our, this is already in there generated it. The middle loads that environment. And then down below, I use a dot gems file with RVM gem sets. And what I do is RVM gem set. It'll RVM gems at import dot gems in that file. I have just one word bundler,

or you could do a bundler space dash v 1.0.15, whatever. And what that does is when I cd into the directory, not only does it load the RVM environment, it makes sure that bundler and the proper version is installed in that gem set. Taking that further, a little bit down more in the generated file, you can actually see that it has a, you know, if bundler is found, then call bundle when you see the end there so that it also bundles for you. Now, that's optional, but so, yeah, so that's RVMRC files and proper usage of them. Cool.

We know you had a hard stop at five, so one last question before we let you go. When you're not hacking on RVM or BDSM, what open source project out there has you excited that you want to tell the world about? And by world, I mean like our 200 users or listeners. 200 listeners. So basically, I'm actually currently working on a specification for a new open source project which does not have a name yet i am working on this with michael and the idea is that it is the system which allows you to write plugins for it for processing arbitrary um data streams or which have the idea of identities and relationships. So there's identities and interactions between identities.

It's a generic framework for this kind of thing. So applications for it could be like monitoring systems, could have a plug-in which basically maps this data and calculates and computes relationships on aggregated monitoring things that come into it. Or you could actually do neat things like business metrics. Like if you can pull in metrics from your ticketing system, pull in metrics from your sales department, pull in metrics from your Nagios as well at the same time. Pull in any and all kinds of different metrics. And basically you write plug-ins which know how to parse these metrics into the format required by the system. And then what happens is when it's in the system, the system itself has a bunch of defaults. It has a graph database inside of it, as well as a document data store, as well as a relational data store, all three of which work together so that basically the entry into the system is the document data store,

and that's where all the raw information is stored. And then the plug-ins are expected to have processors written in them which extract that and store the load the extract data into the idr the relationship the graph database and then inside of the graph database there's your processor will look at that and extract statistics the system will actually extract statistics based on what it sees in the giant graph relationship database and store them into the tables. And then what you can do is you can actually use these things for alerting, for reporting, like business reporting

and reflecting on your business and determining directions and stuff like that. So that's one kind of thing I'm looking at and thinking about. And as well, there's something called event stream processing and workflow engines and combining the two of those together and putting them on top of this system as well will end up with this extremely flexible system. It's going to be generic and the plugins are what actually are business specific or application specific and the applications of this are i've just seen this need in so many different areas

could you do social network activity streams for instance that is exactly correct yes you could uh you could you could do uh let's say you want to have a bDSM relationship dating site and you, I don't know. Theoretically. Theoretically. The canonical example is like if, you know, everybody uses Twitter as a canonical example. The idea is that you would write a Twitter processor which is able to import Twitter relationships. So you have these Twitter IDs and then you have different kind of interaction types between these IDs like follow or retweet or mention, that kind of stuff. And then put that in the system and then it'll compute. Based on the data types that are labeled in the graph database, there are

processors which will extract statistical information and store those into relational database as statistics storage, which can be used for reporting and extracting and whatever the heck you want to do, really. Very interesting. Yeah, I've seen that pattern a lot. It seems to be kind of the pattern behind a lot of web applications and even business applications like you mentioned now. Yep. Well, Wayne, surely appreciate it, taking the time to tell us about RVM and BDSM.

And look forward to this yet unnamed project. Cool. Yeah, thanks a lot, Wayne. Cool. All right, guys. I see it in my eyes So how could I forget when I found myself for the first time Safe in your arms

As the dark passion