The Changelog: Software Development, Open Source - Go Programming (Interview)

Episode Date: August 14, 2013

This episode is part of our remastered greatest hits collection and features Rob Pike and Andrew Gerrand talking about the history and latest updates to the Go programming language.

This episode is part of our remastered Greatest Hits collection and features Rob Pike and Andrew Durand talking about Go. We are joined today by Rob Pike and Andrew Durand from Google to talk about All Things Go, the language that is. So welcome to the show, guys. Hello, Danny. Yeah, we're excited. I mean, we've heard a lot of interest around Go in the last couple years, but it seems

like in the last six months, we've had a lot of people hitting us up, hey, you guys should cover more Go. You guys should get some people from Go on the show. So why don't we kind of get started and talk about just for the people that may have heard of it but don't know much about it, can you guys give us a little bit of an introduction to the language? So, well, Go is a language that was released in November 2009, so about three, three and a half years ago, nearly four. And basically it's a language that sort of sits in the middle ground between low-level languages like C and your sort of scripting language like Python or Ruby.

But it's a statically typed language, so it gives you all of the safety that comes with that. And it has some concurrency primitives that make it easy to model concurrent programs, processes. And a nice system of interfaces, which makes it easy to write composable software. But it's also a very simple language. So unlike a really heavyweight language like C++, which has pretty much any feature you could imagine and they're adding more every day, Go has a much more minimal philosophy where we put only the things that we thought

were really essential into the language. And so as a result, I think the real selling point of Go is that it provides a great developer experience. We try to make everything as consistent as possible. So it's a lot easier to keep all of the state that you need to write software in your head at one time. It's kind of a difficult thing to summarize because I can tell you a list of things that Go is or Go isn't. But really, I think it's the experience that makes Go what it is. The last few days, I've been using Go for a personal project, which surprisingly, I haven't been able to do very much of at all.

But the last few days, I've been having fun doing some stuff. And I need to connect to some old software I wrote many years ago and I had got kind of an A-B comparison of the code I was writing in Go versus the C code from 20 years ago that did similar things and it was an interesting comparison because I remember very much writing the old C version of this stuff and how long it took and how difficult it was and this was a time when I was probably as experienced in C programming as just about anybody but I find that the Go code ends up being a third to a half as long on the page. It takes me an order of magnitude less to get it right. Once it compiles, it's probably correct and

bug free. And so the productivity I've been feeling is really, really high. But then this morning I was noticing when I was running my test, the entire test suite took eight milliseconds to run. And so it's productive for me, but it's also productive for the computer. And I think that's the thing that's drawing people into the language. They just feel like once they finally try it out, they're pleasantly surprised with how productive they feel and how fast everything runs, not just their own world, but the computer as well.

Yeah, so obviously speed is kind of always a top priority, but it seems like a lot of the questions that I've seen just kind of perusing the internet for all things Go related, seems like a lot of the questions I see are things that, you know, why doesn't Go do X or why doesn't Go support Y, right? So as you said, it kind of seems to imply that the language like has had simplicity kind of as a core value. I mean, would you agree with that? Absolutely. But when you ask why doesn't it have something, that's kind of a strange question because, I mean, you can answer it. You have to give me specifics. Why does it have X or Y? But the thing you should be asking is, given that it doesn't have this thing,

how do you expect me to solve this particular problem I have? And we can always answer that question more productively than just saying why some particular feature is missing. The thing is that although it's turned out to be very popular with a lot of Ruby and Python programmers, we came from a much more system side of the world, mostly writing in C and C++. And we tried to reduce the number of things in the language down to a set where we could precisely control the semantics of the language, the semantics of the implementation. We wanted to have a language that we could completely keep in our heads, that any two Go programmers would have the same language they're using. So having worked in C and C++ for many years, especially C++, if I see someone else's C++ program,

three-quarters of the time I have to go look at the reference manual to see what this feature is they're using because I may not even know what it is or I may be unfamiliar with it or I've never used it in practice. That's a really awful state to be in. I've never written a lot of Ruby, but I've heard similar things about Ruby that it's very hard to read a Ruby program. Not that the language is big, but that there's so many different ways you can say effectively the same thing. And we want to go to be much more direct. Not necessarily there's only one way to do it.

I hope a language is never that restrictive. But however you choose to do it, I can read it and quickly understand what it is you're trying to get done. And by me and you, I don't just mean me and you. I might mean me and me six months from now, coming back to code I've forgotten about. It's very important to be able to read code later that I wrote six months ago because I don't have it in my head anymore. Yeah. I mean, when, and I think it's a good point when people ask the question, why doesn't Go do whatever it is, whether they're talking about exceptions or generic types or,

you know, things like that, the answer that the community seems to give them is, well, hey, here's what you're trying to do, so you don't need that to solve this problem. Here's the solution. Here's how you can solve this problem. That's something that happens organically, and the community itself grows up around it with that mindset. I think Go has captured that for systems developers, which is a good, you know, maybe a separation from what the history of these languages has been. So that's a good thing for sure.

I think one thing that's worth noticing and something that people don't think about a lot is that Go places a lot of emphasis on constructing well-specified and kind of pure APIs. So it gives you a lot of language features that are specifically designed to make it like the system of interfaces, for example, the packaging system, the way names are exported or unexported from package namespaces. All these features are designed to make it easy to specify a good API.

And I think that Go really pushes programmers to think about API design. But then once you're actually doing the implementation, it's nice for implementing stuff, but because it lacks some of the meta-programming stuff you find in a lot of scripting languages or even in C++, you might find that rather than being able to don't repeat yourself to the utmost extreme, you might have to copy and paste a bit of code here and there instead of sharing it in some central place. And so I think in some places Go forces you to do a little bit more work on the implementation side,

but I think that the benefits as far as comprehensibility is concerned, far outweigh the drawbacks on that side. So I think that's something that newcomers to Go have to sort of adjust to, is that if they don't have their favorite pet feature from another language that lets them avoid writing the same for loop twice, they kind of just have to deal with that. Right. People coming to go from other languages come with a way of solving problems that they're used to and a set of principles that they believe are

sacrosanct from their environment and go probably doesn't really address either of those but instead substitutes a different way of looking or things or building software and one of the things you have to learn when you move to go is to release yourself from some of those old ways of working, not because they're incorrect, but because they're not the best way to get it done in Go. Go is weird in this respect. When we first announced it back in 2009, there was a fair bit of negativity on the response to the language

because people looked at it and didn't see anything interesting or didn't see things they expected to see. There's a fair bit of sort of negative press about it. There's a fair bit of positive stuff too, but I think it was mostly negative. But over time, people started using it, becoming familiar with it, and realizing that Go is actually surprisingly good because although it doesn't look like there's very much there, what is there combines so powerfully that what seems like a very simple and kind of dull language

when you read the spec is actually incredibly powerful and productive because it has a fairly strong set of totally orthogonal features that combine really easily to build really strong software very quickly. That's something that's very hard to see when you read the spec. You really have to use the language to feel it. And I think the reason it's been catching on lately is that the community of users has grown enough that people are telling one another about, you know, this language you heard about is actually a lot of fun, you should try it,

convincing their friends to try it, and there's kind of a runaway effect going on. And it's finally nice to see that sort of growth happen by sort of word of mouth and people happily talking about this language and how to use it and why it's different, because it's not gratuitously different, it's different for a reason. Yeah, it's kind of what I was hitting on before about the organic growth that Go has kind of seemed to garner, which is like the community is growing it rather than it being forced down anybody's throat.

Which I think the fact that the reason that I, just as some lonely old developer over in the States, the reason that I have heard about Go has been from people that I know and respect and am friends with and work with, recommending that I check it out and look into it. And that's something that, I don't know what it is that you guys captured

to get that with Go, but that's something that I think kind of encourages the community to be more supportive and more integrated rather than this fragmentation that you might see in some of the other languages that, you know, have been around for so long. I think there are a couple of reasons why the language worked like that, that are worth mentioning. It was mostly designed by Ken Thompson, Robert Greaser and myself, who are at least in one case literally a graybeard. We'd been programming for a long time. We tried many different systems.

And we kind of knew what was good and bad in the programming environments that we'd used. And what we tried to do was to reduce it down. We're all minimalists by nature. And so it was very important that we construct something that was small and simple and easy. But we're also pragmatists. We write code that works. We like things that work. And the combination of pragmatism and minimalism is, I wouldn't say unique to Go, but it's

certainly very strongly expressed in Go. Another feature of the design process is that Ken and Robert and I are all very different. You know, Robert comes from a background that's very different from the one Ken comes from. And Ken and I overlapped a little bit at Bell, well, we overlapped a lot at Bell Labs, but our backgrounds are very different. And so we have three very different, one of the essential points of the language design was that anything that went into the language had to be agreed as necessary by all three of us. It was designed entirely by consensus by three very different people. And as a result, everything that went in felt absolutely necessary, no matter where you were coming from, no matter what your background was

or what you thought was important in program development. Now, of course, it doesn't cover the entire space of what people think is important, but I think it really mattered to have such different viewpoints come in and yet arrive at a consensus. I think that's why the language manages to succeed, even though it seems fairly simple. And it's that everything that went in, went in after a lot of thought, a lot of discussion, and a very different analysis from people with very different backgrounds. It's also, you know, it's not supposed to be like a neat, cool language. It's supposed to be, it's very utilitarian in design.

It was designed to do, to solve the problems that we had at work, right? Yeah, and you're not language designers. We're not language designers by nature. We're programmers. We happen to know how to build language implementations. And so, yeah, I mean, we were working at Google. We were unhappy with the development environment we had. We felt unproductive, and we wanted to be productive again.

And the specific problems we were trying to solve were those forced on us by the Google environment internally and the C++ language in particular. I talked at length at this at a Splash keynote last year that you can find online pretty easily. But what was interesting, what was surprising to me, very pleasantly surprising to me, was although we came from that very specific sort of problem space, the thing that resulted seems to be generally useful and very popular in a wide variety of contexts. We thought of it and, in fact, initially branded it as a systems language. But it's pretty clear it's not a systems language alone. It's a general purpose programming language that people have used to do a lot of very,

very different things. Cool. So we kind of took a deep dive into what what the uh what the language is and what's the purpose of it but you know we didn't really kind of talk about you guys at all so if you guys wouldn't mind just kind of circling back and for in the sake of filling in some gaps can you kind of give us just a little bit of an introduction to who each of you are and what your kind of role is with the project well i'm rob. I am a physicist by training, but ended up working at Bell Labs Research

in the computing science department for many years. That was the lab that developed Unix. I wasn't involved in the early Unix as I came in there shortly after Research Version 7 came out, but I was intimately involved in the research editions of Unix that followed Version 8, version 9, version 10. You probably don't know what any of those are. But then very much like with Go,

Ken and I started talking in the 80s about, you know, Unix wasn't able to grow anymore the way that we wanted to, and so we started talking about other things we could think about, ways to sort of rejigger the models that we had and deal with networking and graphics better and so on. And Plan 9 from Bell Labs came out of that work. I think Plan 9 is a very interesting system. It never caught on.

You can argue about why it didn't catch on. I don't want to have that argument. But I think that the ideas in it are still very relevant and some of them have made it into other places. Linux has some of the critical ideas from plan 9 but not nearly enough and then you know we worked on that for a while then you know business markets changed and we had to do something much more commercial we did this system called Inferno which was a sort of set-top box system and an interesting virtual machine language

that ran on it called Limbo, which happened to have the bad luck to come out exactly the same time as Java. And given that Sun's marketing and machinery was much better than AT&T's and Lucent's, it never really saw the light of day the way that we'd like it to. But then the world collapsed because people didn't understand how telecom growth worked. In the early 2000s, I was looking for a job and Google seemed like the interesting place to go. I think that was a good call on my part.

I've been at Google for about 10 years now, working originally on things largely behind the scenes, systems infrastructure, the stuff behind that little tiny text entry box you see, but stuff that I really can't talk about very much because it's kind of how, you know, it's part of the secret sauce, although I was a very small part of the machinery. And then, you know, as I said, about five years ago, we started being unhappy with working in C++, working on 10 million line pieces of code and trying to think about better ways to do stuff. So although I'm not a language designer by nature, I actually solved a number of problems through my career

by designing custom languages. So we proposed doing another one to solve Google's problems, and that's pretty much what I've been doing for the last few years. So what does your role on the actual maintaining of the language itself now look like? I spend a fair bit of my time basically dealing with the open source community, writing. I tend to think of myself more as a library guy as far as Go is concerned. I do a lot of library work, tooling. I've written some of the stuff that will be in 1.2 when it comes out.

But I'm just one of what we call core members of the team here. Google pays me to work on Go, among other things, and it's a real privilege to be able to do that. Absolutely. And my name is Andrew Durand, and I'm an Australian, as you can probably tell from my bizarre accent that Rob reminds me of as often as possible. I started working for internet companies as a teenager as a programmer and have just moved

around in the industry in Australia working in a variety of environments in different languages, none of which is particularly noteworthy, but it was certainly a good experience. And then just around when Go was released, my friend Pamela Fox was sitting near Rob. She was a Googler. Her father was my advisor as a graduate student, by the way. That's total luck. Nothing to do with any of this.

No. But she was in developer relations. She knew that the go team was looking for somebody to be um a developer relations person on go which is someone who's basically a public interface between a google project and the outside world um and she said yeah you should you should get involved with this go thing and i was like i don't know this go language looks pretty weird uh maybe i I guess I could see. And, you know, interviewed, met some of the team,

got really excited about it, got the job, and within a few months was totally immersed in the Go world and having a great time. And so my role in the team is, as I said, to be sort of a public face for the project to a degree. I mean, I give a lot of a public face for the project to a degree. I mean, I give a lot of talks. I write the Go blog. I spend a lot of time arguing with people on Hacker News and Reddit and stuff like that,

just trying to make sure that our positions are well understood as a team, as a project. I work with people in the community. I also maintain a few of the tools, some related to documentation and like our presentation software and also work on parts of the standard library and tool chain and produce the binary distributions and all these kind of like pieces that make Go easier for developers to use. So I'm really just there making sure that we don't lose sight of our users. But it's actually a kind of unusual role for someone in my position at Google, because most people with my job description at this company work with an internal Google engineering

team and the community that uses that product, say like the App Engine team and people who use App Engine. But for me, I work with this group of people who are also in the open source community. So I kind of have... My job's actually a lot easier in many ways because if people have questions about our development process, I can just point them to the development mailing list and they can see exactly how we're doing the work as we're doing it.

There are no secrets, really. Yeah, one thing we haven't mentioned that I think should be made clear here is that Go is a truly open source project. All the development happens in the open. As I like to say, Google imports the open source project for its internal Go development, not the other way around. So it's a truly open source project. If Google suddenly decided they didn't like us working on this and they pulled the plug, there would still be a thriving community of open source developers on the project.

It's not a Google-branded project in any way. It's just that Google likes having us work on it because it helps them for its own development work. But it's truly an open source project. Yeah, you actually hit on something. I noticed that the golang.org website doesn't have any Google branding on it at all. Yeah, they actually wouldn't let us. We wanted to put Google branding on it when we launched,

and they said, no, you can't. And now I realize that was actually very smart of them. It's actually really a good idea. Because it's not a Google project in any formal sense. And we have many contributors from outside Google that work on the project every day. And to put a Google name on it would kind of detract from the huge efforts from the open source community.

There are far more committers outside Google than inside at this point. Far, far more. Yeah, so about four years ago when you guys were on the, when you, Rob, were on the show previously, I think you said that there were six people, maybe six or seven people in Google that were working on the project. So how many people in Google that were working on the project. How many people in Google now are working on the project? I don't know. Not a lot more. Maybe twice that. It's still a very small project from Google's point of view.

But the community itself is much larger that's around, that's for sure. Yeah, there are thousands of people out there and many, many companies. Every morning I get up, there's some new blog that's gone up from some company I haven't heard of talking about how they're using Go internally. It's really gratifying. I'm really personally pleased because I think that we saw certain development problems we were trying to deal with.

And we're a language to try to make them better, which on the face of it sounds kind of crazy. It doesn't sound like the language is the problem. But it is, and I talk about that a lot in the Splash Talk. But it's nice to see that other companies have found the same consequences of switching to Go, that they actually are productive, their software is safe and efficient, and it's actually really rewarding. One of the things that doesn't get talked about at Go very much, I did mention in the Splash Talk last year quite a bit, but it's really the only place,

is that the problems we were facing developing software at Google were problems of development in the large. Software constructed of many millions or even tens of millions of lines of code, thousands of engineers working on it actively, large-scale deployments, many instances of a binary running in production. Those are things that the languages, really any of the languages I know hasn't explicitly been designed to address.

And Go was designed to make that kind of development process more productive. And there's a lot of reasons of, there's a lot of features in the language that are explicitly designed around programming in the large, even though when you're using the language from day to day, you don't really feel it that way. How dependency works,

rules about how naming works in the global space, properties of the production tool chain, a lot of things that really are about making the large-scale software development process very productive. Surprisingly, it turns out that also makes small-scale development much nicer, too. But that wasn't our goal. So your goal, as you've stated, was kind of, you were frustrated with the state of the language that you guys were using internally, so you developed this. And how, inside of Google, how have you seen the adoption of Go in the sense that, you know, what other projects that you can talk about are you guys using internally that maybe you didn't expect to see kind of Go work for?

Well, I don't know. So probably, okay, so first of all um to answer the question about usage i mean goes usage inside google is is substantial and growing all the time i mean understandably we have a colossal code base and it's not like we're going around rewriting it um but there's a lot of greenfield development being done in go um a lot of people are doing v2s of things in go um and it's it's really exciting to see all that work happening. The growth is spectacular. Yeah, it is.

We've got graphs, and they're nice to look at. Unfortunately, it's all confidential, so we can't share numbers or anything. But as far as surprises, it sort of comes back to that original point of it originally being pitched as a systems language. Like Rob and the team were thinking about the problems they were solving and how Go might be applied to those.

But then, you know, we have probably some of the surprising areas are like in our operations teams who built a lot of tools in Python, which was also one of Google's canonical languages. When Go came along, they were like, oh, geez, well, this is way nicer to write tools in because the static typing and the concurrency primitives give them the right sort of palette of tools to write really reliable software.

And so if you're in an ops team and you're writing a tool that reaches out to 10,000 servers and changes the configuration somewhere, I think Go is a much better suited tool for that task than Python. And so we saw a lot of growth in that area very early on because these are really small tools that have a narrow scope and are easy to write in a short period of time. So I think that was a surprise. One thing that's happening lately,

which is gratifying, although it doesn't help Go directly, is that a lot of the libraries in other languages, particularly C++, are being redesigned internally to be more like the Go libraries that we wrote. And I take that as being a vote of confidence in our approach, if not for our language. Because, to be honest, there are systems you can't, for one reason or another, just replace with a Go program. You can't. Some things have too low latency requirements that a Go program. You can't, you know, some things have too low latency

requirements that a garbage cycle language can't handle or there's just too much code that exists already in C++ or Java and you can't just throw it away and start over. But it's nice to see that internally there's been a lot of influence on the environment from Go, even from people who are not using Go itself

and that's pretty gratifying. It's really worth saying that it's not just dissatisfaction with any particular language that drove Go. It was also dissatisfaction with the environment in which those languages existed. And so we're talking about build systems, the way code is organized. And with something like C++,

you can't do what we do in Go, which is Go source code defines all the information that you need to know about its dependencies. And so you don't need make files or Google's equivalent of make files. You don't need all this kind of metadata to actually build projects. And so with Go, we demonstrated that that could be done. But you simply can't do that in C++ or Java or even Python. And so, you know, internally at Google, if you write a Go program, you can run a program that generates the make file

for that Go program. And so you no longer, as a Go programmer at Google, need to worry about a whole section of Google's development process. And so we're sort of demonstrating that your programming environment, your world as a programmer, can be simpler with Go. And a lot of that is starting to catch on. Right.

The standard Google build system is an amazing piece of technology. It lets you build 100 million line C++ programs reasonably efficiently. The standard Google build system is an amazing piece of technology. It lets you build 100 million line C++ programs reasonably efficiently. But it takes minutes. And it involves a cloud build system with many, many servers doing distributed builds and lots of caching of source code and object code. It's an incredibly sophisticated and large-scale system. Whereas a Go program of scale can easily be built on a single computer.

You don't need any of that. And I think that's an interesting sort of data point in this whole thing. Ironically, in order to work inside the Google code base, Go has to run in the distributed build system because everything else does too. And so it's kind of weird, but using Go inside Google 3 is much nicer than using C++ inside the Google world. But it's not as nice as it is using it in isolation because you have to integrate it into this other build system which is designed for languages that don't provide the kind of support

that Go does for efficient compilation. It's still pretty efficient, but it's not the same level of smoothness. Ironically, it can often be faster to just disable the distributed compilation part when building pure Go stuff. If you can get away with it. Although if you're using the libraries to work in the Google code base, you usually can't do that. But still, people are using Go inside Google a lot because it is just so much more productive. Yeah. Can you talk about what specific things inside Google are... I mean, are there anything that you can actually

say is using Go that we might know about now? Sure. Well, a popular example is YouTube. They have a project called Vitesse, which is essentially a load balancing solution for MySQL. They're a really heavy MySQL user. And as you can imagine,

YouTube does a lot of database activity involved in running YouTube. And sitting between YouTube's users, the front-end software, and the MySQL databases is a cluster of this process called Vitesse that basically shards and directs MySQL queries to the correct replicas and so on. And that's an open-source project, so that's why I can talk about it. But that handles some colossal amount of traffic at the moment. And every YouTube page you go to talks to this thing. Yeah.

And it's actually, they were a pretty early user of Go and because of their scale, they really pushed the Go runtime and we made a lot of improvements and fixed a lot of issues early on as a result of their use. I remember one change to the runtime got their system to run eight times faster.

They were a really good user to have. Yeah, and they probably weren't upset about that change either. No, but they needed it because they were having scaling issues, and we had to deliver. Having a real customer like that is a very important part of developing a system like this. Another example is the download server, dl.google.com, which you might not know about, but it's the thing that delivers Chrome binaries to your world, connects to Eclipse, does Android SDK downloads, stuff like that.

It actually handles a staggering amount of traffic. And it was a C++ binary that was quite well written, very clean program, written about 5, 6 years ago, something like that. Maybe even more. Very important program. But what happens in any kind of large code base like this is developers move around projects, code kind of rots because the development environment it runs in changes, libraries

it's connected to change. And so over time, in order to keep up with things, mostly by sort of the process by which, as Brad Fitzpatrick once said, things turn to shit, the code is no longer really capable of handling its job anymore. And a rewrite was necessary. None of the original authors were still involved in the project. And Brad complained on an internal mailing list that, mailing list that this thing is not working very well.

He was doing an apt-get and it was waiting five minutes to receive the headers from the servers. He just literally volunteered, I'll help you guys rewrite it, but you have to do it in Go. After a couple of meetings, they said, okay, we will. Brad joined the team for several months really a month was the core project they completely rewrote it to be completely like bit level feature compatible with the old binary except that it was all written in go it was cleaner

smaller faster uh used less memory had much better latency characteristics and was much easier to maintain and also as a result of this rewrite was able to be adapted much easier to maintain. And also, as a result of this rewrite, was able to be adapted much better to the current infrastructure. And so it now has a very different serving model than it used to, but that came after the rewrite. It was a consequence of the rewrite rather than the reason for the rewrite. But that's a, I mean, again, that's another server running inside Google that handles a staggering amount

of traffic, but is entirely Go. The people on the team are very, very happy with the result. Yeah, one really great thing about that particular project... He gave a talk about it actually. Yeah, he gave a talk which you can find at talks.gorlang.org at OSCON this year. One really nice thing about that project is that because Brad is also the maintainer of Go's HTTP stack, that server uses Go's HTTP stack. So it's exactly the same thing that serves all that traffic.

It's the same thing that you can use in your projects written in Go. And a lot of the, you know, when we deployed it and exposed it to this large amount of traffic, we were able to improve the performance of the HTTP stack and also release an open source project called GrooveCache, which we just published a couple of weeks ago. And that sort of is the distributed caching system for the download server that now anybody can use in their Go programs. You should check it out.

It's github.com.go.lang.groupcache. Yeah, the standard library with Go is extremely good for doing this kind of work. And when dl.google.com was announced as being now on Go, a lot of people said, well, what framework did you use for web development for this? And the answer is, well, we just use the standard library. And a lot of people said, well, what framework did you use for web development for this? And the answer is, well, we just used a standard library. And a lot of people were surprised,

I think, at the idea that a language would come with an HTTP server in the library capable of handling this kind of throughput. But it does. It's probably one of the really nice things about Go is simply when it was created, because a lot of the languages that are around today were written

at least 10 years ago, and their standard libraries were built at least 10 years ago. And the world was actually quite a different place then. The idea of having a, you know, 10 years ago when you deployed a web service, you would put it behind Apache, you know, by default. That was just what you did as a web developer. But now it's totally commonplace to just deploy your own server that speaks HTTP and sort of go through up in that world. You know, we have a JSON encoder decoder in the standard library, you know, that didn't

even exist at the time that Ruby or Python were created. And so Go has the advantage of being more contemporary in that sense. Yeah, if you want to write a web server that serves the directory tree, just like a basic example, it's about 10 lines of Go code for the whole thing because the library just has these rich kind of things. Also, it's got really good crypto

support built into the standard library, so it can do TLS and kind of cryptographic stuff, which is, again, sort of more important now than was thought like 10 years ago. So it's a modern language as much. It's not a modern language in the sense that it has every modern language feature you've ever heard of, but it's a modern language in the sense that the language plus its libraries tend to make it very easy to solve the kind of problems that modern development has.

A little bit earlier, you said that kind of one of the goals was to be able to keep the whole language in your head um is it still that way now would you say oh yeah definitely it's a very small spec relatively speaking um it's about 50 pages when you print it out and a number of people have said they've enjoyed reading it because it's actually like easy to understand just as a document. I usually... Where does document... Sorry, go ahead. I was just going to say, I usually say that you can become productive in Go in a weekend, and you can

become fully confident in Go within a couple of months. There's really not a huge amount to learn. Most of what you learn in the long tail is just idioms and small little tricks and things. And the library. Yeah, and the libraries. But you can become proficient at reading any Go code in a matter of weeks.

Weeks, I think. Yeah, less. Much less, yeah. It depends on how quickly you like it. There's an online resource for learning the language to get you started at tour.golang.org, T-O-U-R.golang.org, which is an online system that lets you run Go code

right in the browser and go step-by-step through learning the basics of the language. It's a really effective tool for getting started. So you said you're working on 1.2 right now, is that right? Yes. Well, we just released 1.1.2, which is the 1.1. It's not very exciting. It's got bug fixes on 1.1.

But it was yesterday, so I've got to mention it. Oh, nice. So the exciting release, from what I've gathered, was the 1.1 release, right? That was a very, very heavy performance increase. Is that right? Yeah, but I think the really important release was 1.0 because if you go to trends.google.com

and look up Golang, you can see this incredible growth that occurred right around the time 1.0 was released. The point about 1.0 was at that point we committed to keeping programs compatible. As we continued to develop, even if we found mistakes we wish were different, we were not going to break your code anymore. Before then, it was a very active process as we futzed around and tried to understand

what the right things were to do. But at 1.0, we said, that's it. The language is designed. The libraries are designed. They will continue to maybe grow, but we're not going to break your programs anymore. And as a result, people were much more willing to commit to using it because they didn't have to track a moving target. And it had a huge increase in the growth of the system after that.

And that was a really, really important step to take. And the run-up to it was exciting because we basically made a huge list of all the things we didn't like. And we went through them. We argued about what we should fix and what we shouldn't fix and did a lot of interesting development work. Andrew and I gave a talk at, I think it was OSCON two years ago, about how this process worked. It was actually kind of an interesting development effort on its own. And so that was a big deal. But then you're right, 1.1 was a performance release.

We got enormous improvement, particularly in the runtime, a little bit in the compiler. Some benchmarks ran massively faster. Typical speedup, I said, was something like 30% to 40% for an ordinary program. And performance work continues. It's getting better all the time. And we expect 1.2 will have at least modest performance improvements for some things.

Yeah, 1.2 will have at least something like 30, 40% improvement for networking on Windows. And there's actually a surprising amount of people using Windows and Go. Yeah, Windows for us was a really good open source story because the entire Windows development work was done in the open source community. We are not Windows developers. We're not familiar with it. And the open source community did all of the work to port Go to Windows,

and it's been tremendous to watch. And I think actually more people use it on Windows than anywhere else at this point. The downloads are certainly really high. Well, I think, interestingly, Go has a lot of usage in China. And I think a lot of the Windows users are based in China as well. And actually one of the main contributors who worked on Windows support, Shenggu Ma, is based in China as well. I don't know how much longer, maybe.

I'd like to move here. Yeah, so the 1.1 release was a performance thing. Some benchmarks saw like 10x, although that was pretty unusual. And then 1.2, a lot of the focus has been on tooling. There's some interesting new tools coming out in 1.2. One of the things that makes Go interesting

from my point of view, not necessarily for everyone's, is that the language comes with libraries that understand Go programs. So there's a parser, there's a lexer, there's a new type checker, which has just appeared. And it's possible to write really interesting programs that do things to Go programs on the fly to rewrite the source code or analyze it and study it different ways. Existing tools like the Go formatter and Go fix and Go doc

are written on top of that. But there's a spate of new tools coming out building on that stuff that are pretty interesting, and a bunch of those will be part of the 1.2 distribution. Gotcha. When do you think that 1.2 will come out? Around what time? It's slated to be released on December 1st, or is it

November 1st? I think it's December 1st. Feature freeze is beginning of September and then we soak it until December. Yeah. I mean, we may cut the release before then, if it's stable enough. So there was a gap of a year or more, 14 months between 1.0 and 1.1 and

we really wanted to close that gap. So after 1.1 we announced a plan to do the next release in six months and that's December 1st. And we really want to kind of even make that a bit shorter and have a sort of three month period of flurry development and then like a one month stabilization period um and then and then cut because go doesn't we don't do any branch development um we maintain a release branch which is you know at the moment that's 1.1.x um but we only apply like absolutely critical fixes to that branch um there's never any new features um and we're very conservative about what we release because we take the stability really seriously. But we sort of copped a bit of criticism for not

having a development branch and a stable branch and pushing more stuff into the stable branch. But really in our experience, if you have that kind of situation, you end up with a lot of people just using either one of the branches and the development one doesn't get enough attention or the stable branch just doesn't get used. And so we kind of have this just single mainline development approach, which means that cutting stable releases is of paramount

importance. We'd like to get to the point of doing three solid stable releases a year. It looks like we're heading towards that. The last release was very painful. It took us about four months of stabilization because after we got all the features in there was just such dramatic

changes to the runtime for performance that there were a lot of secure bugs that cropped up. We also started doing performance work way too close to when we wanted to do it. Yeah, because I was on vacation and it started when I wasn't around to yell. Don't do that. So this time it should be a lot smoother for us.

I don't think the users outside noticed any of this, but for us it was pretty frantic. Gotcha. So if you don't mind mind i'd like to kind of pick some of the questions off of your faq and uh you know about around language specifics and design and i know we talked a little bit about it but i'd like to kind of get into a few of them and just kind of get some uh see what you guys your thoughts are on them um so one of the ones that i that kind of jumped out at me was in the FAQ, I saw you guys had written, it says, although Go has static types, the language attempts to make types feel lighter weight

than in typical object-oriented languages. Can you kind of elaborate on that? What do you mean by that? Yeah, there's a couple of things that are important there. There's a feeling in the community from Python and Ruby World that static types is a bad idea because they learned what static typing is from languages that did it badly, like C and C++ and Java, where it feels like you're filling in some bureaucratic form every time you want to start writing a program. I had a bit of an epiphany about a year ago when I realized that in the last decade, testing has become hugely recognized as a critical part of software development.

And ironically, it wasn't all that big a deal until, you know, there were advocates for it, but it didn't really catch on as an important feature until, you know, the last decade or so. And the realization I had was that was triggered by the dynamic languages community using tests to do their type checking for them, that they'd write these dynamic languages and then write tests to make sure that the strings never turn into lists and vice versa. And it's kind of backwards. Static checking gives you that right at compile time.

Your program won't compile if you try to do that kind of thing. And so that means that the testing can be functional rather than type checking. And that actually makes you have to write fewer tests but get just as good a coverage of your software. So the problem with static type checking is that you want to make it not feel cumbersome. You don't want to write, you know, public static final foo equals new foo.foo of foo.this, foo.that. You want to just say what you mean. And there's one way in which Go does that is it has this notion of initialize with type. So if I say x colon equals and then some expression, that's a declaration of a variable called x whose value comes from the initialized agent value, but whose type is also statically determined by the type of that initialized value.

And so if you want to declare a new foo, you just say foo colon equals new foo, and you're done. You don't have to write foo four or five times on that line to get it to work. It's a little thing, but it has a huge effect on the horizontal width of your program and therefore how many keystrokes, that kind of typing, you have to do. It's a small thing, but it makes a big deal. Another one is the use of interfaces. We've talked about that a little bit.

But I think the most important one is that Java in particular, and C++ as well, were the dominant languages in the late 90s, and the programmers who came out of education then learned to design by constructing type hierarchies. And I think that's a really interesting model for software development, but I don't think it's a particularly productive one. And Go doesn't have a type hierarchy at all. The language's types are not constructed as being inherited from one another.

It's instead a totally flat space of types. And it sounds like a crazy model, but the way interfaces work makes that all work beautifully. And it has a very different feel. You tend to write Go programs based on what you want done rather than what the structure of the type system is for the problem you're trying to solve. And so there's just a lot less structure to a program. This has a big effect on software development because if you want to

write a large Java application, the first thing you have to do is design the type hierarchy. And then you start writing a bunch of code that's dependent on that hierarchy. And if you find out a month into the project that the type hierarchy isn't right, it's actually sort of too hard to change. So you tend to just sort of work around it and go that way. It's even more true in the long term, you know, six months or a year from now, you may need to put a new feature in and it doesn't fit because the type hierarchy isn't right. So you find a way to force it in that's very complicated and it doesn't work very well. And I think that that's just an upside down way of programming. I think they're much better to think about what the functionality is

and develop your software that way and have the types fit into the design of the software rather than the other way around. And Go encourages that model. And as a result, although it's a very subjective thing, I think that Go program design is much more fluid than it is in these inheritance-driven design systems. I think...

Yeah. Sorry. I was going to say the interfaces that you... That was probably the one thing that jumped off the screen at me when I'm reading about Go, is the way that Go handles interfaces is very unique. You know, you don't have to struggle through multiple inheritance. You don't have to struggle through things that are very frustrating.

So that's one of the things I think that's probably so attractive to developers. But, I mean, it doesn't seem like the way that Go handles interfaces is like, you know, it seems almost like a, duh, why aren't more languages doing that? And why do you think

that there aren't more languages come out that kind of find that middle ground of maybe not the type hierarchy that you're used to but maybe not also just all generics. What do you think it is about Go that you're one of the only that seems to do interfaces that way? I think it's because of the people who

designed it didn't think that that way of design made a lot of sense. It's just a bias. I mean, I admit it's a bias. But there's also a kind of orthodoxy about object a bias. I mean, I admit it's a bias, but... But there's also a kind of orthodoxy about object-orientism. You know, you must have inheritance. You must have public and private.

There's a kind of idea about this is how OOP is done. And Go doesn't give you a lot of those tools because they just weren't seen as necessary. Well, actually, not seen as necessary is wrong. Se was seen as harmful. We thought it actually made program development harder, and that's why.

And what matters in object-oriented programming to us is the idea of interfaces. And I've said this many times. I wrote the Plan 9 kernel with a lot of help from Ken, and although it's entirely written in C, it's an extremely heavily object-oriented system in the strongest possible sense because every single computable thing in that kernel or on the network or distributed across the network implements exactly the same interface. It has 14 methods and everything does those exact 14 methods.

And that is where all the power of Plan 9 comes from, is that uniformity of interface. And I think Go, not explicitly, but it's not a coincidence, takes exactly the same approach to the way interfaces work. What matters is not how something does or who its parent is or who its children are, but what it actually does. And the way you say what something does is you write down the methods that it implements and call that an interface. Gotcha. I cut you off a minute ago. Andrew, were you going to say something? Oh, I can't remember now. No problem. I can't say

why people haven't done this. I mean, it's just, that's not for me to say. I hope that, you know, Russ and I said long ago, Russ Cox, who joined the project a couple years after we started, maybe less, he said, you know, if Go dies, at least we who joined the project a couple years after we started, maybe less, he said if Go dies, at least we've got the interface idea out there, and

people will start to pick up on it. And I hope that's true. Not that Go dies, but that the interface stuff... Yeah, I'm glad to know I'm not the only developer that that was a huge sigh of relief for. When I saw that, that just was very exciting to me. Another thing that I noticed, and we talk a lot on this show, actually,

about it's kind of, you know, it's like an ongoing joke amongst a lot of developers, and that's like, or not joke, but I don't know if it's a debate or not, but people talk about exceptions for control flow. And so we've talked about that with a few guests on this show, and, you know, everyone generally has the same idea that that's a horrible idea.

But you guys kind of took it a step further, and Go doesn't have exceptions. So can you talk about that a little bit? Well, I guess it comes down to, you know, what the word exception implies exceptional. And I don't know about most people's programs, but my programs, error states is often the common case. Or 50% of the time or whatever.

I mean, handling errors is what most programs spend a lot of time doing. And so I think Nigel made a really nice point the other day is that there's this kind of continuum of four states where on the one hand you favor... Wait, so there's favoring weak error handling or robust error handling on one axis, and then you have... What is it?

I don't remember this conversation. Oh, well. But anyway. Anyway, the point is that you can kind of be really brief in your error handling and kind of omit it entirely so python is very exception heavy it uses exception to indicate exceptions to indicate a lot of things and that's great if you just want to write a short script and do x y and z and just have it blow up if if x y and z doesn't happen correctly and so on the uh in that case you you know that's really optimized for the

small situation but then if you want to handle any of those potential error states you need to wrap almost everything that you do in a try-catch block and but in go there's only one way to handle those errors and that is just to to write an if statement and check an error return value. And so it means that throughout Go code, you always see the error handling. It's always right there. It's never some invisible control flow that you get with exceptions. I'd like to add to that in two ways.

First, exceptions crash your program, and you can't afford to have servers crashing. It's fine to use exceptions on a page of code if you don't mind if the program doesn't work very well because you're just doing a simple little test. But if you're running something in production, you can't have these stack unwinding things happening all the time. It's very unsafe, unpredictable, and it's just bad design. And in fact, internally, Google does not use exceptions in its C++ development. We just think they're too dangerous. That's an interesting point on its own. The other thing to say about error handling is, touching on something Andrew said,

errors are common. They happen all the time. I mean, you can't open a file. Oops, you know, I shouldn't panic when that happens. I should instead just do something with that result. And so errors are such a uniform and common piece of computing that in Go, we just made them be a value. An error is just a value in the language, like an integer or a string. It's a thing called an error. And it's a computable value. And the thing about the way error handling works in Go is errors are just normal things. They occur all the

time. And you have the entire programming language at your disposal to decide what to do with them. You can put methods on them. You can wrap them. You can do ifs on them. You can do for loops over them if you want. You pretty print them, whatever it is. You have the entire language there to compute with an error value because you need to. You get an error, you've got to deal with it. If errors are handled entirely by exceptions, they're this mysterious background force that you don't get to compute with. You have to use these special control values, special control structures. It inverts your program. They look very strange on the page. You can't just say, here's an error. I want to think about this error for a minute,

write code about it. So Go is actually just really puts a stake in the sand that says errors are ordinary. They should be values. You should compute with them. And yeah, you have to put if error checks every once in a while, but that forces you to think about the errors when they happen rather than throw them up the chain and hope you don't crash. And if you actually do a comparison, a direct comparison between a programming language that uses exceptions and Go that actually handles all the errors in a robust way,

I don't think the verbosity argument really holds up. No, it doesn't. I think if anything, it's equivalent. Often I've seen Go be shorter and much more comprehensible. Yeah. The multi-value returns kind of is the crucial kind of linchpin

to that working, though. Absolutely. I mean, it's not just for error handling, though. I mean, multi-value returns is extremely useful. though wouldn't you think absolutely i mean it's not not just for error handling though i mean multi-value returns is extremely useful yeah that's i i'm a fan as a rubiest i'm a fan of multi-value returns for sure sure um so i think we could we could talk about this like forever i mean i just love hearing you guys talk about go and it's exciting and encouraging to me to get started.

What would you say, quickly, what would you kind of say the future of Go looks like? I just think it's more Go code in more places. The more people use Go, the more libraries people write, the broader its potential use cases are. I don't think there are any real limits as to what Go is useful for. I think where people may have reservations about garbage collection and so on and latency

sensitive environments, I think that all of these problems can be essentially solved. I'd like to see Go on more platforms doing more things for more people. I know that's an extremely broad and vague response, but the reality is that Go is a general-purpose language, and so the vision of Go's future is correspondingly general.

Yeah, and I think it's happening. Ken Thompson and I are astronomy fans, and we've written between us a fair bit of astronomical software, and Ken's been thinking about writing some again and i was looking around on the web and uh i found this package that written i think someone at either the harvard observatory or the smithsonian observatory had written this this suite of astronomical algorithms in go and it's beautiful code it's beautifully written beautifully documented really well thought out and i and it was really a you know very happy moment for me you realize here's a software that I actually want to use

that's written in the world that we built and the open source community has given it back to me and I'm so happy that I can use this stuff to build the thing I'm working on. It's really wonderful to see this happen. But in the near term, I think the real place we're seeing a lot of growth with Go usage is in the sort of DevOps communities.

I mean, one recent Go project that's really exciting is called Docker, which is a software suite for managing Linux containers, essentially. So it's great for deploying and compartmentalizing processes running on servers. And I think these kinds of systems deployment and infrastructure tooling is a real growth area for Go. I think particularly Go's concurrency stuff and its sort of closeness to the machine make it really well suited to doing this kind of stuff. It's like you can write stuff, so you can write the kind of tools

that you would have written in C, except most tools are actually fairly high level. They just need to be able to make system calls and do things that C is particularly good at. But I think Go is really making strong inroads into that kind of sphere. Yeah, Docker's an interesting case because it's catching on.

It seems to be getting a lot of attention. People are really happy with it. But it's all written in Go, and nobody talks about the fact that it's written in Go. And that's great. Yeah, that's great. They're just talking about it as a thing, not that, oh, this thing happens to where you can go.

It's a thing that's great on its own. And I love that. We actually had Solomon on the show from DocCloud about two months ago, and he was talking about Docker. And yeah, he was able to basically, I mean, we obviously talked a little bit about Go, but he was just talking about the power of Docker. And just hearing about a project that's being used as heavily as Docker and in production

in places that Docker is being used and the language itself was not a problem and not something that you had to defend, I think is a big milestone for the language. I think those days are behind us. We don't have to defend it anymore like that. Gotcha. Yeah so like I said I feel like maybe we could we could talk about this for hours but I think maybe that that is for our listeners that are kind of interested in it maybe that that means they need to go sit on some of these talks you guys talk about so much. Right. You'll put some links on the webpage with this, right, so they know where to go to see resources?

Yep, yeah. So, yeah, we will. Do you have any talks coming up that you'd like to kind of talk about? Oh, so coming up in April next year is Go4Con in Denver, Colorado, which is the world's first PureGo conference, large-scale PureGo conference. And Rob and I will both be delivering keynotes there, and they've just opened their call for papers, and so there'll be a bunch of other Go people involved in that,

and it should be a lot of fun. And it's organized entirely by the open source community, which is fantastic. We had nothing to do with this. It's great. That's good. Yeah, no, I'm the open source community. Fantastic. We had nothing to do with this. It's great. That's good. Yeah, no, I'm really excited about that. Awesome.

So for our listeners that are new to the show, we do a thing every show. We ask kind of the same three questions at the end of the show. We'll go ahead and ask them now. Andrew, the first question is for you, and it is for a call to arm. So what in the community would you, and it is for a call to arms. So what in the community would you like to see kind of people contribute to or jump around to, specifically to Go?

I just want people to build more stuff. And the cooler it is, the better. I mean, a lot of the great work in Go tools and libraries is driven by people scratching their own itches, and that's just how it tends to work in the open source community. And yeah, I just, I think, you know, the time to, the time has come to really just build and build. And that's going to be Go's greatest strength moving forward.

It's the community that's building things. What about you, Rob? Anything to add to that? I agree, except I would also add that they should be creative. I mean, Go is a different language. It's a different world. It gives you a different way of thinking. Don't just build the same things you used to build.

Have fun. Be creative. Surprise us with the things you can do. Awesome. So if you weren't working on Go or specifically maybe even at Google, what would you guys see yourself doing? I personally, if I wasn't at Google or working on Go, I'd like to be using Go,

but specifically I'd probably be building music technology, either software or hardware, or preferably a combination of both. Music's been a passion of mine for a long time. It's always been something that's going along in the background, but I think if the next thing comes along, it will hopefully be in that sphere. If I were working on Go

and presuming it didn't exist, I would probably be doing what I used to do, which was systems infrastructure stuff in C and C++ and feeling unproductive and thinking about maybe trying to find a way to do better. Awesome. And our last question is for a programming hero, just somebody that has been influential in your life. And Andrew, you're not allowed to say Rob, but if you want to give us a, someone else. Well, you know, I'm sort of surrounded by people who do heroic stuff all the time, so it's kind of hard to choose one.

And so in thinking about it, I would probably go back to one of my childhood inspirations as a programmer, which is John Carmack. Playing his games and seeing what was possible on those really low-end PCs was incredibly inspiring. And, you know, at that time I was doing graphics programming and seeing what he did just made me realize, you know, holy shit, there's just so much more that you can do with so little. And, you know, even to this day he continues to push the state of the art, so he's truly an extraordinary guy.

He's pretty cool. Yeah, he helped you learn how to cut monsters' heads off with chainsaws. Right. When he was working on Quake, when I was doing Plan 9, we exchanged occasional late-night email. It was really fun to talk to him. It was really good. What about you, Rob?

I have similar privileges. There's lots of people I could mention. I'd like to mention two people that you probably have never heard of that you should know about. One is my old boss, Doug McElroy, who was, when I first got to the labs, I asked around, you know, who are the smart people here? Ken said, well, nobody's smarter than Doug. And he's right. Doug is an amazing guy. He invented a couple of things you might have heard of, pipes, macro assemblers. But he also just had this brilliant way of thinking about stuff

that I found really inspirational. He was the voice behind Unix that never really appeared much in public and never wrote a lot that people saw, although he wrote introductions without attribution to a lot of manuals and stuff. But he had a huge effect on the growth of Unix, and people just love him. He's a retired professor now at Dartmouth, and you go there and you can find accolades from his students and so on.

The other person I'd like to mention is somebody I didn't actually know very well. I knew him a little bit, but I think he was an amazing guy, and he died a few years ago. His name was David Wheeler. He was at Manchester. He worked on EDSAC. Later he was a Cambridge student and was an advisor for some really important people. But when he was at Manchester in 49, 50, 51, he and Morris Wilkes programmed up that thing. They invented subroutine libraries, linkers, assemblers, and all kinds of other stuff. And he pretty much invented computer science as far as I'm concerned.

He later had an office in Cambridge full of filing cabinets and students would come in and say, ìOh, I had this really great idea and here it is.î And Dave would say, ìOh, that is a really good idea.î And he'd go to his file cabinet and open it up and find handwritten notes from 1953 with the same idea but expanded in much more detail and theorems proven and stuff. He was a really amazing guy. He and Morris Wilkes came up with a book entitled something like Preparation of Programs for a Digital Computer. It's a bunch of early papers about the EDSAC. And for those of you who have worked only in the web era, it's really, I think, in your interest to go back and read how clever some of the guys long before that really were.

David Wheeler is an example of that. Yeah, there was a quote when I went to college in one of my uh classrooms there was a quote from i think it was from david wheeler but it's it was something along the lines of you can solve every problem with another level of indirection yeah that's something yeah yeah i remember that that's good so that's awesome well thanks guys so much for coming on the show uh i mean the amount of time and effort you guys are putting into this, I think, is going to do wondrous things for developers all over the world, from China to the States,

or for people down in Australia that say XYZ. That's something that you don't hear every day on the show. We should give shout-outs to the Swedes as well, who seem to be really getting into Go. And I'd also like to give shout-outs to the other members of the Go team, both internally and externally, because Go would not be anything like what it is today without the incredible contributions

of people all around the world, both inside Google, because the team is actually global, even though it's small, and also to the open-source community who've been just so great at making things happen for us. Awesome. That's it for today's show. Thanks so much guys for coming on. It was a pleasure to chat with you guys.

And I look forward to the next time I see you guys. Maybe I can bump into y'all at a conference and pick your brains about what you guys are doing. This is tremendous stuff. So thanks so much for coming on with us today. Thanks for having us. You're welcome. Come to Go4Con. Thanks for watching!