The Changelog: Software Development, Open Source - Vagrant and virtualized environments (Interview)

Episode Date: February 9, 2012

Wynn caught up with Mitchell Hashimoto from the Vagrant project to talk about virtualized environments, DevOps, and more.

The Change Log is brought to you by Pusher, a service that lets you add real-time features into your app. With libraries for JavaScript, Ruby, PHP, and more, Pusher makes it super easy to push data from your backend to your connected clients. Check out pusher.com slash showcase to see what Gages, CloudApp, and many others have already done with the Pusher API. Plus, for this week, they wanted us to let you know they're looking for new members of the Pusherati. Go to pusher.com slash jobs for more details. Welcome to the ChangeLog episode 0.7.2. I'm Adam Stachowiak. And I'm Winn Netherland.

This is the ChangeLog. We cover what's fresh and new in open source. If you found us on iTunes, we're also on the web at thechangelog.com. We're also up on GitHub. And if you head to github.com slash explore, you'll find some trending repos, some feature repos from our blog, as well as the audio podcast. And if you're on Twitter, don't follow ChangeLog Show. Follow The ChangeLog. That's who we are. And if you're on Twitter, don't follow ChangeLogShow. Follow TheChangeLog.

That's who we are. And I'm Adam Stack. And I'm Penguin, P-E-N-G-W-Y-N-N. The ChangeLog, big news. Big, big news. A good Samaritan out there helped us out, Alex Dunney, who is also Mr. Mr. Bug on the Twitter. He gave up that prize ChangeLog, The the changelog handle, so we're excited about

being the changelog on Twitter. So thanks again, Alex. And if you need web dev work and you're in Canada, check out dialect.ca, which is his shop up in the western parts of Canada. And speaking of other good news, we've got a new sponsor

from the guys over at Pusher. We've got Pusher on board. We're excited about having them back the show. We're fans of Pusher and the real-time WebSockets API work that they do. We use it at Pure Charity. Some of our real-time views. Also Gage's app is a big user

of Pusher and some other apps you probably already use. We're also looking to hire some folks on the team to be part of what they call the Pusherati. The The pusherati, yeah. If you're looking to be on the pusher team and if you're a pusher fan, they need some evangelists, one in the mobile space, one in the dev space. And basically, you get to tour the world and have some fun and talk about cool apps and promote open source and all that fun stuff. So check out pusher.com slash jobs or also jobs at pusher.com. Sounds like a dream gig.

Fun episode this week. We talked to Mitchell Hashimoto from the Vagrant Project about Vagrant and virtual machines and DevOps and all kinds of stuff. Distributed virtualized development environments. That's a mouthful. It is. Fun way of saying that you can hydrate virtual machines with little or no work. So DevOps making life easy for the rest of us devs.

We're always on the lookout for how to spin up new environments more easily. This was a fun conversation around virtualized computing and DevOps and some other things. And if you go back in time, we actually did a whole entire episode on DevOps. So brush up on some Chef and Puppet because you can use those too and learn more about Vagrant in this show. Hopefully it won't be the last episodes for all the folks whining about all of the web stuff we're doing on the show. This is for you, DevOps. There you go.

You ready to get to it? Let's do it. Chatting today with Mitchell Hashimoto from the Vagrant Project. So Mitchell, why don't you introduce yourself, a little bit about your background and what Vagrant is. All right. Hello. Thanks for having me. Like I said,

my name's Mitchell Hashimoto. I've been working on Vagrant in my free time for the past two years. And my history is both a web developer and for the past year, I've been a full-time operations engineer. So hacking the servers from the inside. And that's about it. So Vagrant, who's it for and what's it do? Vagrant is for anyone who wants to work in a virtual machine. And you might not know whether you want to or not, so the general use cases are moving web development into virtual machines, testing like Puppet or Chef scripts in a virtual machine that represents production.

And the big benefit is just that instead of running a website on your Mac, which isn't very much like how it's going to run in the real world, you could run it on a real Ubuntu server virtual machine, which is possibly how it's going to run in production, or ideally how it's going to run in production, and that minimizes the surprises. You're running with the same server configurations, et cetera, et cetera. So the install docs for Vagrant, it's a gem to install? Is the whole tool chain Ruby?

It's currently, yeah, it's all Ruby, and it's going to stay Ruby, but it's currently a gem. With the first official stable release of Vagrant coming out shortly, I'm switching to installers and packages. That's going to be the preferred way to install. So no more gems soon. So you mentioned Oracle's VirtualBox. Is it tied just to that particular flavor of VM? It is currently, yeah. It was a decision I made early on because VirtualBox is the first hypervisor I found with a full API, and it's worked out pretty great. What are some use cases that you're using it for? Every day, the company I work for, all the developers use it

to work on the main web application, and in my role, I use it to test our Chef cookbooks before pushing them out to a staging cluster and then again before pushing them out to production. You mentioned Chef and Puppet. Do you have a preference? No, I'm not religious about either side. I happen to use Chef, but Puppet's fantastic as well. So whichever you feel like learning. Anything Puppet does better than Chef, in your opinion?

I'm a big fan of the declarative nature of Puppet's tools because it's kind of harder to grasp your head around while you're programming. I think we program more in an imperative way, but there's much less surprise when you run the actual code on your servers, whereas Chef sometimes surprises you in ways you would not like it to. So what can you do with a VirtualBox VM that you can't do with Vagrant yet? VirtualBox VM? Well, VirtualBox, so Vagrant really makes it simple

for the common cases of working with VirtualBox, so Vagrant really makes it simple for the common cases of working with VirtualBox and provides ways for you to twiddle the knobs a little bit more. But VirtualBox is a very, very highly featured virtualization software. So Vagrant will do basic things like set up networking for you and stuff like that. But if you use VirtualBox by itself, you could really control bandwidth constraints, CPU limitations, hot-plugging CPUs, a lot of advanced stuff that I honestly haven't seen used by more than a handful of people. But it's there if you want it. So pretty much you're wrapping the entire VirtualBox APIs? I don't wrap them.

I give you access through Vagrant's configuration to actually hit them directly if you need that fine-grained control. But I put a layer on top that makes it very easy to configure them. You just say, you know, I want my virtual machine to have this IP address, and I handle all the configuration underneath, including operating system configuration to make it all configured properly you also support multi uh box environments correct yeah yeah so you could uh you could spin up multiple virtual machines to represent a cluster and they could all communicate to each other any use cases for using vagrant in the cloud uh you mean on a server or something? Yeah, pushing these up to somewhere other than your local infrastructure.

Yeah, there's a couple use cases. A common one is actually a lot of people hook Vagrant into their CI systems, like Jenkins or BuildBot or something, and they use it to run their tests, which is pretty neat. I know folks like LivingSocial do this. And then also there's, I'm sure listeners may have heard of Travis CI, which is a cloud CI system for Ruby. It's a really awesome project,

and they use Vagrant as a way to isolate all of their test builds for projects such as Ruby on Rails. Basically any Ruby project nowadays is tested on Travis CI, and they're able to run arbitrary code because they're isolating it all in these Vagrant virtual machines. So the recent Love Travis CI campaign that launched this week, we've been trying to get those guys on the show, but they keep saying it's not ready yet.

I'm not sure what 1.0 looks like to them. It looks like you're nearing 1.0 with Vagrant as well. Yeah, I'm excited. 1.0 should be out next month. It's been a long time coming. It's almost two years on the project now. So what's the determining factor in flipping that bit to 1.0 and quote-unquote production release for you? I think, I mean, I'm at the point where I'm ready to make some pretty radical changes to Vagrant. And I think before doing that, I need to have a stable release. And there's a large group of companies and organizations that have placed their trust in Vagrant as it is now. And it's only fair to them to come out with a stable release that I'm going

to support for some time without giving them any surprise backwards incompatibilities or something. And for the past, I mean, I'm happy to say in the past year there's been maybe one crashing bug ever reported to Vigrant, and so it's been quite stable. Most of the bugs are very platform-specific, like this specific networking option doesn't work when my host is Gen 2 and my guest is Red Hat, like weird things like that. So I think it's time to just flip the bit, stabilize,

and really launch again and show the world what I've been up to for the past two years and what's stabilized to become a pretty awesome product, although I'm biased. You mentioned your backgrounds in web development. Would you consider yourself, I guess, DevOps now? Yeah, yeah, definitely. I enjoy talking about DevOps quite a bit because I think it's a neat, important movement, and I was always full-time dev and now I'm full-time ops,

and I think that's an interesting point of view to go in that direction, and it's been fun. You see that as a rapidly changing field? Yeah, DevOps, it's been changing quite a bit. I mean, year to year, just with Chef and Puppet coming along, and now all these platforms as a service, and just the way we're thinking about the cloud in general, the way servers are built and there's fault tolerance and how you scale. It just changes so much. It's really exciting. In your day job, where do you fit in the application lifecycle?

Are you fully integrated into the project teams, or are you lower down on the stream there? I work for a pretty small startup company. There's only 20 employees overall. The engineering team is only five of us. So there's four main application engineers, and then I spend around 80% of my time in ops and the other 20 doing various housekeeping around the application itself.

So you mentioned CI. I can see the benefit there. Are you spinning up any sort of environments around QA for human-driven quality assurance? No, all our QA is done by our own team, and they all run Vagrant instances locally, but Vagrant's definitely popular, especially for designers, for example, because

designers need to work with the development site, but they don't want to set up the whole environment. So just getting Vagrant up and running so they can just modify HTML and see it run on their machine is a huge productivity boost. Same for managers. So you can have someone on the team set up the environment and everybody benefits from just cloning that. Yep, exactly. What was it like going from web development into DevOps? What do you miss from just regular application code that's not systems-oriented?

That's a good question. I think I'm still in the... I mean, it's only been a year that I've been doing ops, so I think I'm still in the... I mean, it's only been a year that I've been doing ops, so I think I'm still very... There's still a lot that's new, and I'm still learning a ton, so it's all very exciting to me, and I haven't really missed too much on the web development side yet. So I think we'll see in about a year,

but I mean, I guess I miss some of the problem solving. It's just a different set of problems that you have when you're developing an application versus when you're worrying all day about reliability and monitoring and stuff like that. It's kind of a different game, but I wouldn't say I miss dev yet. One of the things that I enjoy about web development is just pleasing users and engaging with users. I guess your users are more developer types. Is that a blessing or a curse? Well, I guess, yeah.

So I'd say it's both, but more of a blessing because when something goes wrong, it's very easy to give them technical instructions to get assistance, which I like, and, yeah, I would say it's a blessing overall, actually. Are we as developers more demanding as a user base? I think a user of anything is demanding, but I would say developers are less so because they understand the work that's involved to make something happen. And a lot of times they're willing to make it happen themselves.

Like Vagrant has almost 100 contributors now. And a lot of times when they have an itch, they scratch it themselves, which is nice. You just returned from Europe. What were you doing over there? Yeah, I was in Brussels this past weekend for FOSDEM. It was the first time I was over there, and Brussels is an amazing city, and FOSDEM is a pretty amazing conference.

I think they said there was over 5,000 people this year, and they covered topics from DevOps to virtualization to X-windowing systems, cryptography, I mean, like the whole spectrum. And it was really neat to just be able to be like, well, I'm going to go see a talk on configuration management, then I'm going to go see a talk on cryptography, because why not? But yeah, the weather there sucked.

It was like negative 10 degrees Celsius. I don't think I've ever been that cold in my life, having grown up in California. So Vagrant's opened some doors for you to speak. Was that the highlight so far? I've been really lucky. my life having grown up in california so vagrant's opened some doors for you to speak is that uh the highlight so far i've been really lucky um in when vagrant was still relatively new i would say in the summer of 2010 it was like six six months old um engine yard uh specifically carl lurch who's on the ruby on rails core and did bundler half of carl Huda? Yeah, half of Carl Huda. And he discovered

Vagrant in a way, and he liked what he saw. I'm not sure exactly what he thought, but he then told Dr. Nick about it, who's now VP at Engine Yard. And then I was in San Francisco, so we kind of met together, and they ended up giving me an open-source grant, which they've been supporting for the past couple of years, which has allowed me to travel and speak about Vagrant and has given me a few resources such as Windows test machines and CI machines that have really helped push Vagrant along. I think a lot of my success is in a large part due to them. I love how transparent you are

about the finances here on the finance page. So pay for a feature, how many takers on that? Very minor features. I put it there just really, the main reason I put it there is because sometimes people will be like, well, I want it to support VMware Fusion. And then I plan on it, but I really do on my own time what I feel is most wanted or what I need. And I like to have the option where it's like, well, if you really want it, you can just pay for it. So I haven't had anyone pay for a major one, and I'm actually pretty happy about that. But it's there just to give people an option. What has GitHub meant to the project development in the last two years? I couldn't, just in general, I couldn't imagine doing any open source project without GitHub because, I mean, it's such

a standard workflow now. Like, every developer kind of understands the pull request system and Git is pretty ubiquitous in the community now. So just being able to say my project's on GitHub opens the door to thousands of contributions, and it's been awesome. Speaking of, 1,200 watchers, almost 250 forks. Yeah. No open pull requests and only eight issues.

Does that mean you're doing something right, or is it just quiet on the Western front? So I actually close handle issues. Since I'm approaching 1.0, I've been really firing through those pull requests. So I get around maybe five issues a day and maybe two pull requests a day and I try to close them

or at least respond to them same day. But since 1.0 is so close, I've been really, really blasting through them. But if you looked about a month ago, you would have seen 50 issues and 10 pull requests because I was slower then. Do you find yourself doing a lot of the enhancements and patches yourself, or do you encourage the community to submit a patch? It matters how serious it is. If it's a serious bug, and I know it's affecting quite a few people, then I'll fix it myself very quickly.

If it's not so serious, I do ask the person, could you look into fixing this yourself because I'm not going to have time. And it's been pretty successful. I get maybe two or three outside contributions a week, which is much more than a year ago, which is much more than a year before that. So I think it's on the right track. Although I'd love to find someone else to work on it.

Like a core contributor alongside me. So you're a brave soul to be supporting a project like this on Ruby on Windows as well. Are you squashing those bugs at Windows, or are you somebody that does that? Oh my gosh, Windows. Every time I think something's going to be easy on Windows, it always surprises me that it's impossibly hard. Ever since the beginning of Vagrant, it's been a challenge to work on Windows. I have to thank the person who started Vagrant with me, John Bender.

He was the first person who was like, I think this should work on Windows. He put in a lot of work to get the initial versions on Windows. Since then, it's mostly been incremental. Windows is pretty hard, but thanks to people in the Ruby community like Louis Levina who does the Ruby installer and he pretty much watches every issue on Vagrant and when it's

Windows related he almost always responds and helps out, it's been much smoother but definitely, definitely not fun What sort of host operating systems are you running on your VMs? The host operating systems, I've pretty much... Oh, guest, I'm sorry.

Guest, okay, guest. So guest, I usually run Ubuntu, so that's why... Is that your favorite flavor of Linux? Yeah, it's the one I'm most comfortable with. I just never really spend a lot of time with the others, so that's the main reason. And yeah, Ubuntu is the only one I use all the time. What other open source projects out there have you excited that you just want to play with? I mean, I've been really into that just due to my job.

I've been really excited about some server software popping around, like React really excites me right now. But in terms of stuff related to Vagrant, perhaps... It can be anything. Well, there is one related to Vagrant, which is really awesome, called VWi, and it creates, you basically give it definitions

and ISO files, and it creates virtual machine images for you. It could create Vagrant virtual machine images, but it could also create KVM images and VMware Fusion images. And just, if you ever use it the first time you ever watch it run and install something and set up a full system from scratch without you touching your computer once, it's like magic happening. You see things typing inside the VM, you see things installing, and then you see it packaging. It's pretty cool. And yesterday was the first day that I saw a video of a Windows machine being set up through that, and that's really crazy to

see. Do you think that's something that will integrate into Vagrant at some point? Definitely, yeah. So after Vagrant 1.0, one of the primary things I want to do is integrate an image creation aspect into Vagrant, since I think that's a missing half that everyone needs, and Vagrant just doesn't do anything about so far. And Vue is awesome about it, but since we're on different release schedules, there's sometimes incompatibilities, sometimes feature differences,

and I want to line that all up. Do you have a programming hero? Programming hero? I used to ask, who is your programming hero? And now we've had so many people say, I don't have one, that I'm asking if you've got one first. Okay, so, well, the programming hero I have,

no one would have heard about. But it's definitely a guy. Well, that's even better. Yeah. He's not a huge open source guy. He's just a guy that I worked with at my first job, and he kind of taught me everything I knew about a lot of things. I mean, about open source, which editor to use, Linux in general.

And I think I'd be a much different person if I never met him. And so I have to thank him. His name's Tim, by the way. But in terms of open source and people who know, I've always looked up to people like Yehuda Katz and Carl Lurch and Tenderlove, Heron Patterson, and all those guys,

because I think it's amazing how much work they do, how much of a change they make in such a large community, pretty selflessly, and that's always been very cool to me. How long has the Vagrant file been around? Since the beginning? Since the beginning? Since the beginning, yeah. I saw a post the other day, I'm trying to track this down, where the author lamented the fact we've got this proliferation

of rake file inspired files out there, and now you have to go into your editor and tell everyone it's a Ruby file to get syntax highlighting? Well, to help with that, Vagrant does automatically, when you create a Vagrant file, put in the Emacs and Vim headers so they set the right file type.

But yeah, I agree. It's kind of annoying. What's your favorite editor? So, I used Emacs for four years, and last month I switched to Vim as an experiment, but I think I'm going to stick with it because I've liked it. I'm almost

sick to say that I like it a lot. I'm a recent convert too. I've been using it for a few months now. I couldn't imagine going back. The other day I had to go into TextMate to do something. I forget what it was. I'm pressing U and I all over the place trying to get up and down and out of the thing. I'm extremely stubborn about the fact that I will. I haven't had TextMate installed in years, and I refuse to do it, even though for some things I know it would be easier just to fire it up, but I'm just stubborn about it. So when can we expect 1.0?

In less than a month. I have an exact date in my mind, but I don't want to promise anything, so less than a month. And with the release, I'm going to do a homepage redesign. The docs are going to be completely redone, so they're going to be completely up to date. They're going to cover every configuration option. Installers are coming out.

I think it should stabilize everything and make a pretty big splash overall. Good deal. Well, thanks, Mitchell. Appreciate you chatting with us. Thanks. I've been a big fan of the changelog for a while, so I was excited when you asked. So thanks for having me. Thanks again to our sponsor, Pusher, at pusher.com.

They're doing some real fun stuff around hosted APIs and the real-time web. And we also want to thank you for listening to the sponsor, pusher at pusher.com. They're doing some real fun stuff around hosted APIs and the real-time web. And we also want to thank you for listening to the show because without you, this show would not be possible. And if you're interested in sponsoring The Change Law,

we certainly appreciate your support. Shoot us an email at sponsor at thechangelaw.com. Until next time. I found myself for the first time Safe in your arms

As the dark passion