The Changelog: Software Development, Open Source - Luvit and Lua Bindings for libuv (Interview)

Episode Date: May 23, 2012

Wynn caught up with Tim Caswell to talk about Luvit, his new project that provides Lua bindings for libuv.

This episode of The Change Log is brought to you by Hacker Newsletter, a weekly newsletter delivered every Friday that shares some of the best articles on startups, technology, programming, and more. All links are curated by hand from the ever-popular Hacker News. Well, two big events are almost here, the 100th issue of Hacker Newsletter and 10,000 subscribers. To celebrate, 10 lucky subscribers who open the 100th issue will win some very cool prizes. For full details, subscribe today at HackerNewsletter.com. Don't push me away! Welcome to the ChangeLog episode 0.8.080. I'm Adam Stachowiak. And I'm Winn Netherland.

This is the ChangeLog, where we cover what's fresh and new in open source. If you found us on iTunes, we're also up on the web at thechangelog.com. We're also up on GitHub. Head to github.com slash explio. Find some trendy reposts, some feature reposts from our blog, as well as the audio podcast. And if you are on Twitter, follow The Changelog and me, Adam Stack. And I'm Penguin, P-E-N-G-W-I-N-N. We have some news to announce.

You're recently hitched. Yes, I'm married. Heather Stachowiak, she is my bride. Congratulations to you and your lovely bride, Heather. Is she going to be changing it to MRSAdamsStack on Twitter? Is she going to keep her maiden Twitter handle? She's actually already moved over. She's Heather Stack. She's taking the brand.

Nice. Stack attack. Yep, that's true. Heather Stack on Dribbble as well. She's a talented designer in her own right. Yeah, she is. Fun episode this week. We talked to Tim Caswell, a.k.a. CreationX. He's been a contributor for the Change Log back in the early days when Node was the new hotness, but now it's the old stodgy.

So if Node's not cutting edge enough for you, he's got Lovit, which is basically V8 with Lua bindings, if you're so inclined. That sounds like some madness. It is madness. Lua looks like a nice little language. I was exposed to it in Adobe Lightroom, which my wife uses for photography all the time. But it's making its way into every aspect of development now. So we talked about Lua and just what makes it nice.

And then got into WebOS a bit since Tim was previously at Palm and HP. Wow, you guys covered a lot. That's true. Fun episode this week. Should we get to it? Let's do it. Chatting today with Tim Caswell, former contributor to the ChangeLog, big in Node circles, but today we're going to be talking about Lua and Levitt. So, Tim, for folks that might not know you, why don't you introduce yourself? All right, so, as Wynn said, my name is Tim, and I've worked on Node and open source software and web development for a very long time. Even going back to the 90s, I had a small startup making DHTML-based games that you could market in your.com website to bring in traffic. And everyone said I was crazy. They used to be Java applets.

But, well, we saw how that turned out. And since then, I've done everything. I work a lot in Node. I wrote the How to Node blog, and recently I've been experimenting, taking Node's core library, porting it to all sorts of other runtimes. Fresh coat of paint on the How to Node blog. I just noticed that.

Well done. Yes, yes. I didn't do it. It was community-contributed and looked good, so I took it. That's the best way to do it. So let's talk about Lua and Lovit. So how did you get tangled up into Lua? So working on node on webOS, I realized that V8 is not as lightweight

as you would think. I mean, sure. Compared to like Rhino, it's a pretty fast engine because Rhino starts up the JVM, but running V8 on a mobile phone, you realize that it's actually pretty heavy. Just starting up a node process is a minimum 10 megs of RAM. And on the phone, over 1,000 milliseconds of startup time. I mean, that's just, that's crazy. And so I was looking for something like JavaScript, but faster. And I heard a lot of people saying Lua is a good language and the VM's really clean.

And so just for fun, I wrote libuv bindings to Lua to see what Node would be like in a different language. So Lubbock is essentially bindings for libuv. What attracted you to Lua as a language, if anything? I mean, mainly I just, I heard I heard good suggestions from other people TJ liked it and looking at the language, it's a very clean language, it's it was developed independently from JavaScript yet they're extremely

similar, both have closures both have first class functions both have these things like objects and lower they're called tables they hold arbitrary values they even both have the same annoying number type where it's always a double i mean the languages are surprisingly similar there's there's a few core things where they diverge but as far as power and flexibility they're almost identical so is it typed? Nope. The type system is the same as JavaScript. It has coercion and all that timing.

Basically, you take JavaScript, remove the braces, and put an end keyword, and you're pretty much there. So the first time I came across Lua, I guess, was with the Adobe Lightroom product that my wife uses for photography. Plugins were written in Lua. I guess the main app is written in Lua as well. What other main installations have you seen for Lua? So Lua is primarily an embedded system. Its goal is to be embedded. So the game industry has really picked up on it.

Like it's the scripted engine to World of Warcraft. And most first-person shooters will use it. I mean, lots of game engines use it because it's trivial to embed, and it lets you use the scripting language for your logic and your plugins, and then you can use C or C++ for the rest of your game. So where do you see Lovit going? Any practical applications, or is it still just a research tool right now? So it's actually used in production by the people at Rackspace.

And the reason they went with it is because it has a much lower memory overhead than Node. But it has more or less the same API and power of Node. So if you're not just diehard in love with JavaScript, but you want something like Node, you can use Lovit. It's much more efficient on the system. And also, I was thinking that it would be great for

making games. One thing that LuaJIT has that's really amazing is it has built-in FFI. This means you can call out to shared libraries, DLLs, SOs, without writing bindings for them. So I can just write a pure Lua script, and I can make OpenGL calls if the system has OpenGL installed. And so distributing that game is just a script. You don't have to send any binaries for all the various platforms, and it's really fast the way that it's embedded into the VM. So what libraries are available out there already?

I know in Node early on, that was kind of the bottleneck of getting support for other libraries out there. So I changed the module system to be slightly different than Lua because Lua already exists on the server. People use it for web servers, but they're very not Node-like systems. And so I actually changed the module system, but most Lua modules can be ported trivially. You just change how they export as long as they don't use any blocking I.O. And so I actually changed the module system, but most Lua modules can be ported trivially. You just change how they export, as long as they don't use any blocking I.O. As far as things written specifically for Lovet, I know there's a few database drivers. There's Redis.

There's some basic web frameworks. It has built in all the stuff Node has. So we have OpenSSL, Zlib, JSON. Because JSON is actually not part of the Lua language like it is JavaScript. So we had to bind in Yagel as part of Lovit. How far have you gotten? I know with Node, you start seeing these second-level frameworks crop up like Connect and Express. And now there's dozens of these web frameworks that are built on top of it.

Have you gotten that far with Lovit yet? Not much. Rackspace doesn't actually use the HTTP. They just use TCP and JSON, and they're the biggest user of Lovit. I mean, their stuff is they've contributed OpenSSL and Zlib and all these libraries that they need. There are a few people who play with Lovett for web frameworks. I don't know of any actual website using it, so there's not a lot of demand there

yet. There's, I think, two or three mini frameworks for HTTP. What has developing Lovett taught you about Node or JavaScript or V8, perhaps seeing it from a different angle. So, right. One of the reasons I did this experiment was there was this raging argument about how callbacks are difficult. And this argument comes back regularly, I've noticed. And Lua has coroutines built in. They're part of the language. And using these coroutines, you can make code where you call an async function

and then suspend your stack, and you get resumed when the async function calls back. So you don't have to nest. It'll look blocking. And I experimented with this, and yes, you can do that and love it, and I still prefer callbacks. Why is that? I mean, I don't know. Maybe it's because I have so much more

experience with them or I, I like explicitly knowing when I'm between stacks and technically you're not between stacks with coroutines, you're just pausing stacks, but either way, these are the points where you can get preempted or not preempted, but things can change up from under you. You have concurrency. And I don't know. I like callbacks. The biggest dig I have on callbacks, you know, in the JavaScript community, it's almost a four, sometimes an eight space tab set, right? And you start doing these deep nested callbacks.

You almost get to where you're doing nothing but indentation if you want to put a code snippet in a blog post just because you can't see the actual line that's doing the work. Yeah, and there's a lot of techniques to make that better. You don't have to nest. I needed to write a HowToNode blog article about that because that comes up all the time on the Node list and

starts all sorts of flame wars. Also, I use two space indents, so that helps. Are we seeing a backlash in the Node community that perhaps Rails enjoyed after it peaked in popularity that this is just a healthy... It means you've arrived when you hit that Gartner lifecycle spot where people are starting to write articles against Node. There's definitely been negative articles for at least a couple of years. I mean that that started happening pretty early on the community has definitely grown a lot though there's a lot

more people who just use node because they want to get their job done i've seen people who say my employer says i have to use node and i don't like it how do you help me out i mean you don't see that when a project is early on not too much of that that though, but there's definitely a lot more people using it who just want to get work done and not people who want to play with the technology. So beyond V8, you're also playing around with SpiderMonkey. You've got LoveMonkey. How similar are these projects? So LoveMonkey is libuv bindings for SpiderMonkey. And for those that don't know, libuv is the C library that powers Node. It was written for Node 8, and it has full Windows support built in. And that means that most of Node's logic got

ported into the C library. And so that's why I'm able to rebind Node's core to all these other VMs so easily. And so basically, the LoveMon, the goal there, or the LoveMonkey port, the goal there is to be an API-compatible version of Node using SpiderMonkey. Since it's the same language, why not expose the same API in the end? And that way we can have some of this

browser, or not browser engine, but JavaScript engine competition that's good for Node and good for the community. And also people can experiment with some of the JavaScript features that are upcoming at Harmony because SpiderMonkey already has them. What features of SpiderMonkey do you like over V8 and vice versa? The V8C API is definitely easier, or at least cleaner.

The SpiderMonkey code, you can tell that it's been around a long time. But as far as the JavaScript side, I mean, ECMAScript versus JavaScript 185, if you looked at them, you wouldn't even know they were the same language. Because in SpiderMonkey, you have let, you have argument destructuring, you have these expressions, generator expressions. You have generators. I mean, it's code written for SpiderMonkey that knows it's SpiderMonkey. It almost doesn't look like JavaScript to me because I'm used to ECMAScript from being cross-browser compatible. What's your take on some of the recommendations

for the next versions of JavaScript and some of the new features that have been proposed? I like most of the things going into ES6. I'm trying to remember. I think generators are going to be there. Destructuring is going to be there. Let should be there. I'm not too keen on some of the more heavy abstractions,

like a module system for JavaScript itself or a class system. I don't think the language needs these things. But some of the smaller things I like. Some of the things that maybe Lua already has or JavaScript 185 already has. What's your take on AMD? I don't like it.

I mean, it's useful, I guess. When I want to package a node program for the browser, I will just precompile my scripts and add the proper wrappers around them in my generator. And if I only want to write it in the browser, I just write it for the browser. I mean, I don't see what the use case for AMD is. If you're going to involve a node server anyway, why not write it in the browser. I mean, I don't see what the use case for AMD is. If you're going to involve a Node server anyway, why not write it in the simpler style and have your server generate the right code for the browser? Because anything in production, you're going to want

compressed and concatenated scripts anyway. So many folks don't know that you were involved early on with CoffeeScript when you were working at Document Cloud with Jeremy Oshkosh. So talk about Candor and perhaps maybe how CoffeeScript influenced Candor. So, yeah, I worked at Document Cloud with Jeremy. That was great. And I was making this language called Jack, which was basically a simplified JavaScript with a few things added. And then I saw he was making this language called CoffeeScript.

And I said, hey, we're working on the same thing, let's join forces. And so I wrote the first version of the CoffeeScript compiler written in Node, and there were a lot of things I liked about the language. But since then, the language has gotten a lot of features. It's kind of suffered from its own success, and I feel it's a rather bloated language now. Kender, the goal there is to take JavaScript, remove the warts, and make a very bare-bones, very simple language that's very easy to optimize. Kender has its own VM. It's all written in C++.

And the goal is to make a faster VM by making it easier to optimize language. For example, in Kendo, there's no prototypes. There's no new keyword. You can't put properties on functions. Only objects can have properties. It's a very simple language. There's no this magic property. When you call a function, the arguments you pass in are the arguments you get.

So if you're distributing your own VM, you're definitely not targeting this for browser apps then? Right, well, I did start a project called Cander.js where the goal is to make it a transpiler to JavaScript. I just haven't had time to actually work on that. Have you played around with Go at all? A little. It's interesting. I think I like Rust better.

I don't know. These new systems languages, I just want to see where they go after a little while. I definitely agree with a lot of the concepts. The nice thing about Candor is it's the same syntax as JavaScript. It almost looks like a strict subset of it. And I'm assuming it's untyped, same way that JavaScript... Right. Right. So you're working at Cloud9 these days. Yeah. What's going on at Cloud9? What's exciting over there?

All sorts of stuff, trying to get the cloud-based IDE to be faster and more stable and better features. And a lot of that is refactoring the backend. So what I'm doing right now is I'm writing these node plugin systems so that their massive code base can be stable and modular and testable. And we should be ruling that out fairly soon. So for those that don't know, Cloud9 at C9.io is basically a text editor in the sky, but it doesn't suck like you think it might just hearing that. Right. It actually took me a while to get the vision of it. If you look at it as just an editor

in your browser for editing your local files, that doesn't make sense. Why would you do that? But what if you didn't have a local system? What if your files lived in the cloud as well? Because, I mean, GitHub already hosts your Git repos in the cloud. Why not clone them to another place in the cloud, edit them from there, and then push them back to GitHub? And when you look at it that way, it makes a lot more sense. Because it has a console. You can run git commands from the browser. You can push to GitHub from the browser. You can run a server from the browser and point at it from another browser and see you're running web server.

And so you don't need a local machine at all. And once you look at it as moving my dev machine to the cloud, not just moving my editor, then it makes a lot more sense. So the ACE editor that powers Cloud9 IDE is the same editor you get on GitHub when you edit this file? I believe so. If it's syntax highlighted, it's the same one. Reading the blog post here on GitHub. So if you've ever done fork and edit this file, which folks have done all the time with correcting spelling errors on a lot of the apps that I've got, then you've used that feature.

So how big is the team at Cloud9 now? I'm remote. It's hard to tell. I think, I don't know, roughly 20. And how many of those are in Red Lake, Texas? I am the only JavaScript developer within 100 miles. Just the one. I told Tim before we started recording to watch our accents as we text and start talking to each other.

The draws come out. So what's next for you at Node? Have you moved on or are you still flinging Node? No, I still work on Node. I have a new blog that I do as part of my Cloud9 job called NodeBits. And initially, it seemed to overlap heavily with HowToNode,

but actually they are different sites. HowToNode teaches you the theory behind Node. It teaches you what objects mean in JavaScript, how to do callbacks, how to do async. And then NodeBits is sample projects, innovative examples. My most recent article, for example, is how to read the Linux joystick kernel device from Node and parse the joystick events

so that you can interface a joystick with your Node server. So really things that get people thinking outside the box. So the purpose of Node bits is to keep the innovation flowing, keep people having fun with Node. Have we boxed in Node as primarily a web framework? Is it suited for command line apps? It certainly works for command line. Desktop GUI apps are still hard because all the GUI frameworks have their own event loop and those don't meld well with Node's event loop.

That may change in a future version of libUV. The one nice thing about working at Cloud9 is that's where Bert and Ben work and they're basically the libUV guys. So I get to find out the inside scoop of where it's going. And hopefully with some future refactors, you'll be able to have better bindings with like QT or Cocoa and make desktop apps in JavaScript. So some folks might not know that we're both prior HP folk and you were over in the former Palm team and working on, I guess you weren't working on webOS,

but you were involved in that group. What's your thoughts on how that was handled and what future, if any webOS has? I don't know. I mean, when I, when I joined webOS, they were already dying and I knew this,

I knew this going in. I believed in the idea of HTML as the platform. And so I joined the team anyway, as an effort to try to help them. And I did a lot of work optimizing things, optimizing WebKit, and in the end it just died anyway for silly business decisions. The perception on the outside is it's this wonderful operating system that really suffered from the lack of industrial design and the hardware on which to run it.

Is it fair? No. There was a lot of technical debt. There were a lot of issues in the code that the people left in the engineering team could just not handle. When Palm went under and got bought by HP, most of the good people quit. I mean, there were certainly good people left,

but there was way more work than they could do on their own. And we just missed deadline after deadline. I mean, it doesn't help that the CEO does crazy stuff like he did, but on top of that, there were lots of technical issues too. So what's next? What's got you excited that's on your radar other than LibUV that you just want to play with? Speaking of HTML in mobile devices, Boot2Gecko is really interesting.

And webOS is now open source, so maybe that'll get a second life. But I've been playing with Boot2Gecko, and one very important difference between the two is with Boot2Gecko there is no such thing as a local app. Even your dialer and your home screen are just web pages hosted on geomobile.org. The way this works offline is they just pre-fill the cache on the phone, the HTML5 cache with the files for these particular apps. But everything on the phone is a web app. And even your phone dialer. So perhaps a really bad metaphor, this would be like Chrome OS,

except replace Chrome with Mozilla for your phone. Similar, yeah. Except Chrome OS is basically a browser. Whereas these look like native apps. It has a lock screen that you swipe away, and the interface can look like any other phone. But the technology behind them is, yes, they're all just web pages. So there's some similarities.

It's kind of like Chrome OS, it's kind of like web OS, but it's some similarities. It's kind of like Chrome OS. It's kind of like webOS. But it's different too. Well, thanks, Tim, for joining us and keeping us up to date on the Node community and what you're doing. And I hope you'll stop by soon. All right. Thank you. See you next time. For the first time Safe in your arms As the dark passion