The Changelog: Software Development, Open Source - Elixir and the Future of Phoenix (Interview)

Episode Date: February 9, 2016

José Valim joined the show to talk about Elixir. We learned about the early days of José's start as a programmer. José took us back to the beginning of Elixir and shared why Erlang got him so excited, we broke down features of the language, we talked about functional programming, concurrency, developing for multi-core systems, we talked about the Elixir community, the future of Phoenix, Ecto, and more.

I am Jose Valim and you're listening to The Change Log. Welcome back everyone. This is The Change Log and I'm your host Adam Stachowiak. This is episode 194. It's a big show today. We got Jose Valim on the show. We learned about the early days of Jose's start as a programmer. Jose took us back to the beginning of Elixir and shared why Erlang got him so excited. a limb on the show we learned about the early days of jose's start as a programmer jose took us back to the beginning of elixir and share why erlang got him so excited we broke down the features of the language we talked about functional programming we talked about concurrency developing for multi-core systems we talked about the elixir community the future of phoenixcto, and so much more. We had four awesome sponsors,

TopTal, Rollbar, Linode, and TrueSight Pulse. Our first sponsor of the show is our friends at TopTal, an exclusive network of top freelance software developers and designers. Top companies rely on TopTal freelancers every single day for their most mission-critical projects. At TopTal, you'll be part of a worldwide community of engineers and designers with the flexibility to travel, blog on the TopTile engineering blog, apply for open source grants, or for scholarship options for our fellow women developers out there. Lots of opportunity inside of TopTal for you. Head to TopTal.com, that's T-O-P-T-A-L.com to learn more or email

me at adam at changelog.com if you'd prefer a personal introduction to our friends at TopTal. And now, on to the show. All right, everyone, we're here today talking about Elixir. And Jared, you know, we've wanted to have Jose on the show for so long. We had several issues come up. And Jose, I think you and I might have exchanged some sort of GitHub message way, way back in the day, trying to get you on the show, and you were still working more so on Ruby. Yes, indeed. But it's been a long time coming, my friend.

A lot happened in those. I don't remember when we talked, but when I look back, there are so many things that happened in the last five years, which is just crazy. And Jared, like any good show, it begins with an issue. So how did this one come about? Yeah, multiple issues, like you said, over the years. I think the last time around was like the end of 2014, and we couldn't quite schedule it out or line it up with Jose, but we had Chris McCord on the show to talk about Phoenix specifically, which was a great episode.

If you're interested in that, check out 147, which was like last March. And that kind of accounted for our Elixir show for a while. And then recently, in December, Jose, one of your counterparts there at Plataformatech, George, help me with the name, Jose. George Guimaraes. Yes, so shout out to George for hollering again and saying, you know, we've had some big milestones in Elixir 1.2, or is it Ecto 1.2 and Elixir 1.1? I can't remember the exact version numbers.

And it's time to get him on. And yeah, Elixir 1.2, Ecto 1.1. So thanks, George and Jose. We're really happy to have you on the show. Thank you. I'm glad to be here as well. You had Chris back in March last year? Yes. Yes.

Oh, so we have a lot to talk about. Yeah, lots changed in Phoenix as well, right? Yes. Yes. Oh, so we have a lot to talk about. Yeah, lots changed in Phoenix as well, right? Yes. I think March, it was not even 1.0 yet. And I think March was when I was actually starting to contribute to Phoenix, or I had just started. And then we had this great, let's say, sprint where Chris and I were working and we're improving things. And I think we got like day one about July, something like this, kind of middle of last year. Yeah, so we have a ton of catch up on. Adam and I are quite excited about both Elixir and Phoenix.

So we have tons of questions for you. Jose, one thing we like to do kind of as an intro to the show is to get to know the people behind open source because we find that's super interesting and enlightening. So from the internet and from following you and your work, we know a little bit about you, which is that you're a Brazilian and you live in Poland. We know that you crank out a ton of open source stuff. You also seem to be, just from my experience online,

a super nice and a positive person. But we don't know very much about beyond that, really, of your background and how you came to be where you are. So we love to hear developer origin stories on the show. And so if you're willing, we'd like to hear your origin stories on the show. And so if you're willing, we'd like to hear kind of your origin story, where things began for you in software, and then how did you get where you are today? Oh, okay. So I would say things like my first real contact with software was at university uh i went to do so i grew up in the center of

brazil in a small city called in yours and i moved to sao paulo which is you know big city and everything uh for studies to do my university in which end up being automation and control engineering which i don't use for anything but anyway anyway. And the first year I had C programming and that's how I started with programming. And I have very good, and I have very good memories from those classes because I remember, you know, you would, the professor would program in the whiteboard, which is a little bit weird, but he would say, now you need to declare this variable, like int, the variable name. And he would say, you need to do this. And then every time he said, like, you need to, or you must do this, I would ask why, what would happen if you don't? And,

and he, he, he was like very, he was a little bit, uh, peculiar, the professor, but he would always like entertain my questions. But at some point he got like, uh, he couldn't take it anymore. And then he like, why do you want to do it? You're a software. Like you need to follow those rules. Yeah. Why are the rules in place, man?

Tell me. Yeah, exactly. Right. I just, I just wanted to know what was going to happen and and you know and but he he said that like it was also interesting because you would say like oh the variables would need to be initialized right but you see if you don't initialize uh something is going to happen it's just going to have whatever it is in memory. And those weird questions actually made his, made him like tell this stuff as well.

Right. Like what are, what are the consequences if you don't do something? So I think we could learn and hopefully I was not just the annoying person in the classroom getting in the way of, of teaching. But that was, that was pretty much my first

contact with it. Then, during the first year of university, me and a couple friends, we had a band. It was an acoustic band. One of the players of the band with me

is Hugo Barauna, which is a co-founder of platform attack with me so we we had like a lot of story together that started exactly in the first year university and we had a band and i decided to make the website uh for our band using flash and action script nice so this is like the first time i was like, hey, I'm going to learn things for myself and I'm going to try to make this work. And I was doing it out of, you know, it was not because I had classes, even if I enjoyed like the C classes,

I was actually doing this because I wanted and I wanted to learn. And that went pretty much like that. I also remember that at some point I went from, I started learning more about databases, you know, MySQL probably at the time. And then I ended up going to do PHP and MySQL. It was a very short period of time. I did a couple of projects as a freelancer. I remember like a couple interesting stories as well because we were still in the band i was always very passionate with music i remember

that some point when i was still in university i went looking for like music schools like i don't know if it's conservatorium in english or how you say that. And I remember that I was checking on the internet which ones had really horrible website. And I called them and I was saying like, hey, if you give me like classes on singing or guitar, I'm going to do a new website for you. Like if you give me six months of classes, I'm going to do a new website for you.

So that's something that happened at the time. Did that work? Work, it worked. I got my six months of singing, I'm going to do the website for you. So that's something that happened at the time. Did that work? It worked. I got my six months of singing classes, yes. So you're basically trading websites for educations. Sorry? So you're basically trading website work for an education in singing.

Yes, exactly. I couldn't afford the singing classes at a conservatory. Those are usually really good. But so, you know, that was a plan that worked. They're like, well, we would really appreciate a new website. So, yeah. So, you know, so it was pretty much that, nothing serious, just doing things on the side, even because the engineering university was, you know, required a lot of time. And it was at the end of 2006, I had, we were a couple of friends. And that was when I actually met George that got us here on the show. It was about 2006 that we got closer. And then we had some ideas for startups. And Rails was already, you know, a lot of people were talking about Rails and I decided to try it out.

And that's when I got started with Rails. And yeah, and then a lot more happened regarding that. I was doing Rails for quite some time. And at the end of my university, they have an agreement between, so at that point I was still in Brazil, but the university had an engineering, the engineering school had an agreement where I could go on and do my last year of university in another country.

And that's when I moved to Italy and that's when I left Brazil. And it was really funny because when I left, I was like, the whole course was like two years that I, what I had to finish. And then I was thinking, well, you know what? I'm just going to go stay there six months and then come back and not even going to do the whole two years. And then I never came back. I'm still here.

Yeah, but that's how I went abroad. And that's kind of what explains how someone from Brazil is living in Poland. I met my wife. My wife's Polish. And now I live here. I'm living here in Poland for five years. I was going to ask what kept you in Poland, but then you told us you found a wife and settled down. For reasons. Congrats on that. Yes, exactly. Thank you. So you mentioned you were doing Rails work, and many people, I think, probably in our audience who know you and may not yet know you with regard to Elixir

probably know you with regard to the Rails work that you did, which started off as Devise, right? Or maybe that's not the starting point, but that was the gem that you and your team at Plataformatech built that became kind of one of the de facto authentication tools that people use on Rails even to this day. Can you tell us about that section in your software career with regard to device and working with Ruby and then eventually on the Rails core team

sure so uh yeah it was a little bit before that and it's there is a very nice story here because uh i remember my first open source contribution which we had this was probably back 2006 2007 and uh i sent we had a plug-in called upload column if I remember correctly for a os and I remember sending a patch by email to the altar like hey, what if we did those changes and It's really nice because I later, you know, the author, the owner of that package is Jonas Niklas.

And, you know, he went to write Capybara. He wrote Carrowave and the new refile plugin for Rails. And this goes like way back and probably 2006, 2007. Like we were exchanging these emails, we were exchanging patches. And it's nice because recently he started coding with Elixir as well. So that's a fun side story. But that's like what I remember as like my first, my first like dabbling at open source. And a couple of years later, I think it was 2008, 2009, I actually created something called Inherited Resources, which was, I don't know if you ever who are doing Rails for a long time as well,

I don't remember the time, but we also had something called Rails Footnotes, which was a plugin that you had to, you could install in Rails application, and it would add a bunch of footnotes at the bottom saying, showing like what was the request parameters, what was in the logs. So it gave access to a lot of information. It would show like which queries ran and how much time it took. And I also contributed to that. But the first one was exactly inherited resources. And with inherited resources, so this, if I remember correctly, was 2009 and we had

the Google Sermon of Code happening. And this was when Rails 2 was starting to become Rails 3, right? The work towards Rails 3 had already started. And Google Sermon of Code was happening, and I was still a student at the time. So I wrote a proposal for the new generator system, which I still think is the generator system used today in Rails. And the whole idea of the proposal was, you know, Rails 3 is meant to be agnostic and everything, right? Like you can bring your own ORIM layer, you can bring your own like test framework.

But then I said, like, we cannot really say that Rails is going to be agnostic if the generators, they are still going to generate only active record stuff, right? Like at Rails 2, if you're using RSpec, you need to use like RSpec scaffold, RSpec model. They could not play together. So I wrote this Google Sermon of Code proposal for the new generator system, and it was accepted uh i worked with yahooda cat on that he was my mentor and that's how i started to it was a really great opportunity because um you know contributions on github was not that easy at the time like how the it was new still so it was hard for you to be really in touch you know like with the people actually building the

software and it was really hard because i got really close to yahooda we became good friends be really in touch, you know, like with the people actually building the software. And it was really hard because I got really close to Yehuda, we became good friends. And that's how I got like my first big contributions to Rails. And then, you know, I started contributing more and more, eventually became part of the Rails core team. It was also at the time that we started Devise. And Devise, it was started as part of Platform Attack. We hired at the time our first person. In 2009, the company had just started. So we were the four founders. And we hired Carlos Antonio. And

one of the reasons we hired him was exactly because of his other contributions to other open source projects. And he started working on Devise and working together. I was kind of more of doing a mentorship role in those initial days. But then Devise grew up, a lot of people started using it. And it's our biggest open source project for the Rails community in particular. And we have other ones like simple form and so on. Yeah, I actually had forgotten about inherited resources. I recall it now. And as you spoke about it, it kind of reminded me of what you were mentioning

back with your professor in college asking why. Because, correct me if I'm wrong, wasn't the purpose of inherited resources was to really dry up the controllers quite a bit inside Rails and remove a lot of the boilerplate, which is scaffolded out in a typical Rails controller. And it just seems like that was you basically asking why with regards to how we do controllers in Rails and kind of your answer to a different way of doing that.

Yeah, so the whole idea is that you would inherit from, inherit resource space or something like that. And it would bring the whole the whole, you know, it would have the full implementation that does everything the Rails couple is supposed to do. And this is usually, like, what your controller typically does is that you know, you need to have, like, the does is that you need to have like the query part where you need to say if I want to get

the current manager for this project inside this company, you need to build the query, right? And often using things from sessions. So that was one thing. The other one is how to render those resources. And that's kind of what was in there

together and eventually you know at this point if you go to the inherited resource project they're going to say hey don't use this we don't want this anymore we don't recommend people to use it exactly because it just hides too much uh we we you know we we if at some point you say like ah this scaffolding rails there's a lot of boilerplate, and the heritage resources has none, we had to find a balance somewhere. And the heritage resources was too much to the extreme, to the point you would look at our controller and you're not really sure what it is doing. Or you would have to customize like one of 20 callbacks, and then you actually have not a lot of confidence of how that works. Yeah, that's actually my exact experience. I had one situation with it.

I actually inherited a project that used it. And it took me a really long time just to figure out what was going on, because where was all this activity coming from? And then once I realized it, it started to make more sense. But again, like you said, it's funny. The question is why. Why are we doing this boilerplate? And it's a super useful experiment to like,

let's see if we can just dry this up and not have to do it. And then over time you learn, well, it's also helpful for it to be obvious what's going on. And so you're hiding a lot under the covers there. So yeah, interesting stuff. And so Devise came later and was, like you said, part of Plataformatech. We're going to go to a break here real quick, but can you just give us a real quick synopsis of your company, Plataformatech? You said you founded it with

four other people and its purpose and kind of how it plays into open source. Sure. So Plataformatech, we are a consultancy based in Brazil. I said like 2009, we were four, but now we are about 40 developers. And we work with both well-established startups and big companies, Fortune 500, for example. And what we're really good at is to go there. And if you're having hard problems to solve, right, we come in and we're trying to make your whole process around the software development more efficient. And the relation with open source was exactly,

when we started, I was doing open source. So, you know, we started us four and for a while we didn't have any clients. So I had a lot of free time. And that's when I wrote Inherited Resources. And what happened at the time is that we started to get a lot of clients because of our open source work.

So we knew at that point, like, hey, investing in open source, besides, you know, it also helps us to grow the company. And as I said, the first person we hired was also because of the open source contribution. So it's not, we can get clients, right. We can also, it helped us to attract a good talent. And, uh, that's how it, you know, was our relationship with open source. But it changed a lot with Elixir, I would think, because with Elixir, it was when we decided to make a huge bet, right? When you say like, hey, I want to invest in the language,

we are no longer talking about making a small project for the community. You're talking about investing in something for a long period of time that has also higher risks because you can invest for like three years and nobody uses it, right? So what's going to become of that? Well, you did a great job there

teeing up the next segment of the show, Jose. We obviously want to dive deep into that big bet you mentioned, which is Elixir. So we're going to take a break. You know, great learning about your origin and all that. I guess it will definitely dovetail quite deeply into your passion for multi-core and Elixir.

So let's take that break. We'll dive deep when we come back. I'm excited to tell you about our new sponsor, VAR's Rollbar. One of the most frustrating things about being a software developer is dealing with errors. Relying on users to report your errors, digging through log files trying to debug issues, or a million alerts flooding your inbox and ruining your day. With Rollbar's full stack error monitoring, you get the context, the insights, and control you need to find and fix bugs faster with a lot less noise.

Rollbar is easy to install. You can start tracking your production errors and deployments in 8 minutes or less. And Rollbar works with all major languages and frameworks, including Ruby, Python, JavaScript, PHP, Node, iOS, Android, Elixir, and more. Integrate Rollbar into your existing workflow, send error alerts to Slack or HipChat, or automatically create new issues in GitHub, Jira, Asana, Pivotal Tracker. And we have a special offer just for you, our listeners.

Go to rollbar.com slash changelog, sign up and get the bootstrap plan free for 90 days. That's basically 300,000 errors tracked for you, totally free. Rollbar is loved by developers at awesome companies like Heroku, Twilio, Kayak, Instacart, Zendesk, Twitch, and more. Give Rollbar a try today. Head over to rollbar.com slash changelog. All right, we're back from our break. We got Jose Valim here. Long time in the making this show, as many shows, Jared.

Yep. It came from an issue, but it goes much deeper for you, Jose. You came from Ruby Roots, and you you came from roots where you kind of got, I don't want to put words in your mouth, but it seemed like you kind of got bummed about the lack of multi-core systems and concurrency and all these other things that other languages bring. And obviously we've got Elixir now. So maybe let's begin with that. Tell us a story about how it began for you and Elixir. Where were you at with Rails? Where were you at with Ruby? And what kind of sparked this interest of multi-core, concurrency, and ultimately Elixir?

Yeah, that's a great question because I was working with Rails and I was one of the ones responsible, not responsible, but working with making Rails thread safe. And it was really, really hard, you know, and we worked a lot on making or improving Rails. So if you go back in time, like Rails 2.3 said that Rails was finally thread safe, but it was thread safe by putting a huge lock around your application, right? Which is not what you want because you're not going to leverage concurrency.

And then we wanted to improve this more and more time. It was a lot of work. And then when you're when you're thought like, hey, I can finally make this work, then you realize that it doesn't work on JRuby or Rubinius because they give you different guarantees regarding thread safety. So, you know, it was really frustrating work. And as you know, like, if you're using threads and motaxes and so on, and you have a lot of state around, which is what we have in our regular Rails applications, sometimes you don't know that there is a race condition

or there is a concurrency bug in there. It's just when you're running in production under certain scenarios or in particular some high loads, right, that those issues are going to show up. And then they are even harder to debug and try to give a guarantee to say for sure, hey, this is thread safe. So I was working with that. And then what came to my mind was that, you know, I was already hearing, this was, I think, 2010. I was already hearing like, you know, concurrency is becoming even more important. That's why we want to try to save in the first place, right?

So we could get our Rails application, put it in a server in production. And if it had like four cores, it would use all the four cores efficiently without needing to restart for instances, for example. So I knew that, I knew it was becoming important, but I said, you know, like, if this is going to be the future, right?

If the future is going to have like eight cores, 32 cores, or 128 cores, we needed to have better abstractions because the ones I had working with Ruby and Rails, they were not going to cut it. So I decided to study other languages and see what they are doing. And the idea was exactly to kind of see

what is happening there and try to bring it into Ruby and into Rails. And I did, I spent a good period of time studying other languages and so on. But the one I really, really loved was Erlang. And the reason why I really liked Erlang, as not only the language, but also the whole virtual machine,

was exactly because to me, they were no longer, they were not doing the concurrency. They're not worried about concurrency. You're actually worried about distribution, right? You're not writing software and you're saying, oh, I want to make this software concurrent. You're saying, hey, I'm writing this software

and this software can be distributed, which means it can run on multiple machines, right? It just happens that, you know, our model, when you're running multiple machines, it can also happen that you can run everything in the same machine and you get concurrency for free. So I'll expand a little bit on that. When you're writing software in Erlang, and now today in Elixir, all of our code runs inside the processes. And the Erlang virtual machine is like, it's 30

years old. It has been there for a while. And when they were writing it, they were not worried about concurrency, the multi-core concurrency at the time, because they didn't have multi-cores at the time. But so when you're writing Elixir code, you have your code running in processes,

right? And those processes, they're not operating system processes. They are very lightweight threads of execution. They are very cheap. You can create literally millions of those. And it was everything done in a way where, you know, I can have a process running this machine, I can have a process running on another machine.

And as long as this machine, they are connected as part of our cluster, they can exchange messages between them. all those processes they're running at the same time. And that's what they built at the time. And then when they needed concurrency, they just realized that concurrency is the special case where you have all those processes, but instead of them running in different machines, they're only running the same machine. So you just, you can get a little bit extra guarantees for that. But it's really a special case for the model they have that. And when I saw that, I found it beautiful, right?

Because if you think like we as a software, you know, the industry is changing, right? Now we are hearing more and more languages oriented towards concurrency. It may get at some point that we're going to hear more and more languages oriented towards concurrency, it may get at some point that they're going to hear more and more languages that are oriented towards distribution. And Erlang is already there. It has been there for 30 years, right? So that was really fascinating to me.

And I was like, you know, if I want to write software in the future, I want to write software that's going to run on this virtual machine. And that's what got me excited and led me down this path. So just to give us some context, what year was this when you were formulating, you were conceiving the idea of Elixir? Give us a time period. This was 2010 and the beginning of 2011.

So the first commit to Elixir was like beginning of 2011. So the first commit to Elixir was like January of 2011. And it was the time I started. So after that, I started writing more and more Erlang. But there were a couple of things that I really feel like it was missing a language. The way I like to sum it up is that I liked everything I saw,

but I hated the things I didn't see. I wanted, for example, really good Unicode support. I wanted good abstractions for working with collection. Things that I was used to, right? And I could not really give up on them. So after, yeah, go ahead. I was just going to say, it sounds like, you know, the programmer happiness angle that Matt took with Ruby, you know, it spoils you when you get used to it.

I know I'm spoiled in many ways by it. And I look for those features everywhere else I go. And I judge other languages in terms of semantics and syntax with regard to how I can express myself. And it sounds like you were hitting that same thing where like everything you saw about Erlang, the foundations, the distribution model, all these things were great and you loved it. But there was just missing pieces that you just didn't feel like you could live without. Because if you could have lived without them, you could have just kept writing more Erlang, right? But you decided, no, I'm going to actually start something new that's going to be kind of a melding of these two worlds. Is that fair to say? Yes, that was kind of the, it was not the idea at the time, but it's what it came to be.

So at the beginning, so like I say, the first commit was January of 2011. And so I knew what I was missing, but I was not sure what I wanted, if that makes sense. So for example, I was like, oh, I want better support for collections, or I wanted a way to do polymorphism. So the first, if you go to the early commits of Elixir, to the early history, it was actually an object-oriented language. It had a prototype-based model. But everything in the early virtual machine is immutable.

So I was trying those things. So I know I had a problem and I wanted to solve it. But the answers I had at the time, they were very Ruby-centric, let's say. So it was more biased towards object orientation. And however, Ruby was solving those particular problems. So for example, metaprogramming, it was still, it was a metaprogramming similar to Ruby, where, you know, you, when you need to do faster things, you need to be doing classic volume strings and things like that. So I knew I had like those problems and I was trying to solve them. And then I played with it for three or four months.

And the end result was really, really bad because, you know, I was like, I know I had those problems. Those are the solutions I know. And they didn't map really, really well. Right. In the sense that the things I was trying to bring doesn't going to fit in this new ecosystem, this new way of doing things. So I stopped working on Elixir at the time.

And I said, okay, I know those are the problems. And I know that some of the solutions I'm looking right now, they are not going to fit. So I need to study more. I need to see how other languages, they are solving those problems and how they can fit into this new, into this virtual machine. And so that's why, that's when I started to say, okay, so I want,

I don't want like Ruby in Erlang. I actually want, you know, I want to solve those particular problems. And if I need to take some ideas from Ruby, I'll take some ideas from Ruby. But if the best ideas they are from Python that will fit here, they are going to be from Python, they're going to be from Haskell, or they're going to be from Clojure, and so on. And, yeah, go ahead. Yeah, I was just going to say, that's very interesting. It kind of reminds me of what Jeremy Ashkenaz did with CoffeeScript back in the day,

which was, I'm not just going to do a nicer syntax for a Ruby version of JavaScript. I'm actually going to pull in what he considered and what the community considered best ideas from all these different camps specifically, but that Python plus Ruby was major influencers. So I mean, that's definitely a winning strategy. It's like, let's just not take Ruby ideas and move it over here to Erlang world. Let's actually come up with the best thing we can for each given circumstance. But that seems like a really big green field

and there's so many decisions to make. Was it daunting? Were you intimidated by this task that you had just taken on? That's a great question because I think a lot about it. And one of the things that makes it really easy is that if you're building software if you're building a language on top of the early virtual machine there are a bunch of decisions that they are going to be taken for you okay so that narrows a lot the scope of what you can do which is which is know, it helps you, it helps keep you sane, right?

Otherwise it would be too much. I already feel like, you know, being a engineer, being a software developer, I want to build the best software I can possibly build, right? And sometimes I have like those things that, you know, I see something different, I see something new, and I ask, should Elixir be using that instead? Right. How, how, how could, you know, how would that affect the language if I had made this other decision? And those things, they can be quite consuming because also if the answer was yes yes life was going to be much easier because say

okay i know how to make this better but the answer is not yes it's maybe you need to consider how it interacts with the other you know um all the other things that you have in the language right i say like it's like a jenga game you need to have like the pieces well together. And sometimes you can take a piece out and put a better one in place. But maybe you do that and you're going to continue building on top until you find out that, oops, that piece was a bad piece. So I get that a lot. And I say joking that it's something I would never do again, create another programming language because of all those questions that you can have. It's just such a wild scope and you cannot really, you know, choose how like this is a hundred percent better than the other.

But the early return machine helped a lot. So I could get a lot of concerns, a lot of things like, okay, even if this is cool, I know I can have it here. And that's fine, right? That's life. And I also, when I decided, so after I studied and I decided to give Elixir another try, I had a better foundation of what I wanted. So I said, for example, okay, if I'm building for the Erlang virtual machine,

one of the things I want to do is to leverage this virtual machine as efficiently as I can. So I made a decision to, you know, I want to stay. So when you're compiling a code, you have like many compiler steps. And I decided to target a compiler step that was semantically very close to Erlang. And that would give me, that would add even more constraints, right, in language development. So that helped a lot. I also had already made decisions on what I wanted the syntax to be and that I wanted to have a macro system based on AST and so on. So all those things that were like initial decisions, like this is what I want to be

on top of, helped make it a little bit less daunting. So did you just go into a cave for two or three years and write code? Or I mean, you obviously were taking influence from other places. Were there any books, was there any influence where it's like, I'm going to write a programming language. Sounds like you had at least a second start there, which is always nice to throw away those first efforts and start over when you have a little bit of solid

foundation. But what was the process like from, maybe not to 1.0, but here's a new guidepost. In 2013, there's a post by Joe Armstrong, who I'm sure, Jose, you know, but to the audience, he's one of the designers of Erlang. And he published a post about Elixir, I think it was called One Week with the Elixir, in which he said, and I quote,

this is good shit. So I think that feels like a milestone to me. From conception in 2010 and a false start to a certain degree or a restart in 2011 to 2013 you know one of the inventors of Erlang is impressed with what you've done here can you can you walk us through that time period and and the process you went through to to create it sure so uh so at the beginning of 2011 was when I made the prototype. It sucked and I threw it away. And then it was the ending of 2011, beginning of 2012, where I had like, I've built a conceptual model that I said,

okay, I think this can work. And this was also the period where I went to platform attack and say, okay, I think we can build this thing. And I think it can be useful because at the time we were looking at, uh, what I said was something like, you know, concurrency is becoming that thing. I just said, concurrency has become more and more important. And today, if you, if you want to use a language that, uh, for better or worse is a dynamic language and it focuses on concurrency and it focuses on productivity and being expressive, the option we had at the time, the main one, was Clojure. So if you get something like, you have Ruby and Python,

but they are still not concurrency-oriented as those other languages. You have Go, but it has a static type system. They are not very focused on being expressive as a developer, right? So I said, like, the only language that has everything that I could think of us using at the time was Clojure. And I said, you know, we need to have more options, right? And it would be really nice to have an option that runs on the Erlang virtual machine. And that was January 2012.

And it was when the company made the commitment to, okay, we are going to like to invest part-time, half of our time into developing this. So that helped a lot, right? Because it was not me being, you know, coding at night or something like that. It was part of my job from that moment. And so I had this idea of what I wanted the language to be. So as I said, I wanted to be closer to Erlang. And I knew I wanted a macro system. I knew I wanted for polymorphies, for example,

I had completely ditched the idea of objects at this point. I ended up implementing something that's very close to Clojure protocols. And it's similar a little bit to type classes in Haskell. And a little bit of interfaces in Go is kind of the same idea. So that was like, okay, this is what I wanted to build. And then I started working on it. And I don't remember how much time it took me, maybe six months to have something like, hey, I kind of feel comfortable now for releasing Alexir, what I would say like the 0.5, which is what people use when they say like, okay, this is probably usable up to some extent.

And I also made the plan, I think it was even the beginning of the year, to speak at Strangeloop. They had at the time an emerging languages track. And I went there. I was able to speak about Elixir. It's probably one of my first public talks about Elixir. And at that moment, some people already started to find it. So I always had access to a good part of the community. Like a lot of people follow me on Twitter and stuff like that.

So I was always talking a little bit about it. And some people, they decided to try it out and they decided to explore the language. And some people started contributing back. So we see a little bit of a community forming and important contributions to the language that came about that time. So, for example, one of my favorite features we have today is the idea of doc tests. So the here doc syntax we have, like if we want to do a very long text, right? We got it from Python and then someone said,

you know, Python has this doc test thing. It's already got the here docs. Look at the doc test, which allows you to write documentation, or tests inside your documentation. Then you can guarantee that the documentation is up to date, right? And that's something we got at the time. It was a contribution. So the language was growing a little bit.

You know, I was pouring my time. The community already started to help it grow and help it move forward. And it was, I think, so, and then it was, that was 2012, right? I was working on it. The company was investing in it, but I was always a little bit uncertain, right? Because, you know, like I know the company was investing in it, but for how long are we going to actually invest in it? Like if we spend two years working on this and nobody's using it,

does it make sense? So I always had like the feeling that, you know, maybe next year it's going to be my final year working on this. And yeah, it was, but it was like, you know, if that happened, it was like, oh, the ride was fun, right? Like I learned a lot and people, you know, use it and enjoy it.

So that was worthwhile. But it was in 2013 that we had like two very good news, you know, in a row, which was Dave Thomas. He sent me an email asking if he could publish a book about Elixir. And a little bit actually before that, also Simon St. Laurent from O'Reilly, he wrote Introducing Erlang and he said, I want to make this Introducing Elixir 2, which is probably the first Elixir book announced. I don't remember which one of those two they became. And it was just fantastic, right? Because now there you know, effectively investing on the language as well, right? And, you know, like, one of the things I had in my mind is that I didn't want to write a book on Elixir.

I felt like, well, maybe this is something that it's probably going to be necessary in the long run. But I don't want it to be me. But if nobody does that, uh, I, I, I would probably do it, let's say. And, you know, and then Dave Thomas come and say like, Hey, I wrote, write the book. Like, that's great. Right. Like Dave Thomas, he's, he wants to write the book. So that was very exciting. And that was a big deal. Yeah. And that was what led, I think,

was Dave Thomas that led Joe to do that because Joe, he got to know that Dave Thomas was writing a book on Elixir and he was like, okay, I need to try this. And we started growing more and more from that moment on. So, Shirley, you read Joe's post and

you saw the quote, you saw his take on it. Can you recall your thoughts and feelings with regard to Elixir when you see one of the designers of Erlang saying that it's good? Oh, it was really great. I don't remember exactly, but Joe and Robert, they're both creators of Erlang, they were always very open and we could always have conversations.

Also, the OTP team at Ericsson that maintains Erlang, they were always very open and we could always have conversations. Also, the OTP team at Ericsson that maintains Erlang, they were always very open. So I remember when I first announced it, Elixir, the tagline was, Elixir, a modern approach to the Erlang virtual machine. And it was a horrible tagline, but the modern word made some people mad. And I remember getting an email from someone at Ericsson saying, you know, it's fine, ignore that, just continue building with stuff. So that was very encouraging. And getting Joe's feedback and later Robert's feedback and so on,

it was also very encouraging to continue doing the work. Right. I just want to point something out here for the, for the listening audience, just in case they didn't kind of hear what I think I heard, which was here's you and everyone sees you as this really great software developer, especially with all the success with Elixir.

And they see you to what you just said, basically, was that you got a couple years into this and you thought it might not work out. And then you sort of hit this stride where I'm sure one of your heroes, Dave Thomas, who wrote the Ruby programming book, which is the most bought Ruby book ever, I'm sure. So here's one of your heroes, blessed basically what you're doing, and then want to write a book about it. How that all worked out to me is just like a really triumphant moment for you. Yes, yes.

It was those things that give you a lot of confidence, right? To continue moving forward. Right, you're going the right way. Yes, it takes the, you know, the uncertainty, which helps a lot, right?

Removing that uncertainty that I may not be working on this next month, right? So that was really nice. And, and, you know,

I, and the, the benefit is that I became closer to Dave Thomas as well. So when we had the first Alexei conference, it was in Austin and my flight was from Dallas where he lives.

So we took a car ride and were able to talk a lot there. Sometimes I'm like wondering things. I know I have like the perk of being able to send like Dave Thomas and he may even say like, hey, what do you think about this? Right. So those were very nice perks that came as consequence of the book as well i'm gonna tease something up for you jose before we head to this break i'd like to hear from you what some of the overarching features of elixir are when we come back from the break

we'll dive a little deeper into some of them if we can. But do us a favor, share kind of an overarching feature set for Elixir. Okay, yeah, so that's really hard. So I like, I like to do it. Yeah, so Elixir, so the main thing is that you're going to use it to build maintainable and scalable applications. But we, I, we also, Elixir is also an expressive language. So we want to allow that thing, like developers, they can get the language. Actually, I'm not going to say expressive. It can be expressive as well, but extensible.

They can get the language. There's always on the ideas language to be able to get it and extend it to whatever domain you're working on. And the focus on productivity as well. So we have very good tooling. So I would say like the, so that's like the macro overview

of what you can build with the language. And then features, there are so many, but I think like the tooling is really fantastic. I like features like focus on documentation and what else i like our uh the way we can do polymorphism which is a language specific trait in this case and so on i yeah i'm not sure if i that's a good breakdown but yeah that that's what we're looking for i mean we're getting ready to go into this break so we wanted to kind of get a a breakdown to leave the listeners hanging so to speak a little cliffhanger so to

speak on what uh elixir is built upon like and I think the the plain language in which you put it in wasn't like here's the feature list of of uh elixir it's it's more like you know from your from your mouth so I mean from in your own words it wasn't like this scripted list cool uh jared anything you want to add before we go to the break no that's good i've i've been writing those down and we have a list of things that we can definitely use as jumping off points to talk about uh on the other side cool let's take that break we'll be right back our friends linode are huge fans of the show and many of the developers that work at linode listen to the show they're huge fans what that work at Linode listen to the show.

They're huge fans of what we're doing here and they want to support what we're doing. And we want to invite you to try out Linode, one of the most fastest, efficient SSD cloud servers on the market. Use our code CHANGELOG20 to get $20 in credit, basically two free months. Plans start at just $10 a month. They have eight data centers spread across the entire world, North America, Europe, Asia Pacific, they got hourly billing with a monthly cap on all plans and add-on services, you get full root access for more control, run VMs, run containers or even your own private git server, you can enjoy native ssd storage 40 gigabit network intel e5 processors again use the code

changelog20 to get a $20 credit with unlimited uses tell your friends it doesn't expire until the end of this year so use it as many times you want share it to everyone you know head to lenode.com slash changelog to get started. All right, we're back from the break, and we got this feature list, and we're looking at it, and one of them stands out more than the others, and it's maintainable and scalable applications. How do we quantify that?

What exactly does that mean? How do we break that down, Jose? So that's a very good point. I don't like to say, oh, what is the language feature? I like to say like pattern matching, because unless you use the functional programming language, you don't know what it is. So I started with things like, what is the focus? And the focus is maintainability and scalability. And one of the things about maintainability in my experience with Elixir so far and why I think it's a maintainable language is all behind the idea of processes we were talking about. So, for example, if you think about maintaining an application, there are two aspects of it. It's running production and part of maintaining it is to ensure it

actually runs properly in production. So if you never, I know you're already playing with Elixir, but if you never run with it, use it before, we have a tool called Observer that you can install your node or connect to a node and run this tool. And it kind of showed the whole tree of our system. So Elixir applications, so sorry, your Elixir software, your Elixir system is built into a bunch of tiny applications and you can go one by one

and introspect those processes, those lightweight threads of computation we were talking about earlier. So you have really a great amount of introspection of how your system works. And this matters a lot to ensure the software is running properly in production, which is one of the aspects. Just to give an idea of how this is useful. So one of the things that we did with Phoenix and the whole channels idea and WebSockets idea

is to make sure we got a very powerful machine for Rackspace and we're able to have two million connections on that same machine. So we had like two million clients connect to the same machine, connect to Phoenix channels, and we broadcast, for example, a Wikipedia article to those two million clients

in like two, three seconds, right? And in order to make it go that fast, we had to improve it. So what we did is that we connected the clients and then when we, for example, at the first try, we had, what, 300,000 clients where we reached our first bottleneck or even 30,000. I don't remember exactly, but what we did is that we connected the observer.

And because it gives you this whole idea of the system, right, we could say, and because all the code run into processes, those light-ray trends of computation, when things started to go slow, we went to a pain and say, wait, which process is the one that is slow, the one that's doing a lot

of work? Because that's the one that's going to be my bottleneck. And I say, oh, that's the one that's being slow. Let's optimize it. And then we optimize, we move the bottleneck elsewhere. And we did this a couple of times. And I think over the period of two days, we were able to go to two million connections just by relying on this tool.

And these we use as an optimization job, which is also part of maintainability, right? If things are slow, you need to go and try to understand your system and try to make it fast. So that's one aspect that we have to it. The other aspect that we have of maintainability is also related to code. So we were talking about this during the break. So for example, I think the whole idea regarding immutability helps a lot with that. And the whole idea of functional programming.

So, okay, let me just do a statement and then we're going to explore a little bit more. I think a lot of the reason why XS software is more maintainable, it's because of ideas that we have that come from functional programming. And functional programming, you know, if we ask someone like, what is functional programming? You're going to get a bunch of different answers. So one of the things that's usually associated with functional programming is the whole idea of immutability. And I'm going to expand this soon.

But to me, the big thing about functional programming is that it pushes part, it tries to make the complex parts of your system explicit. And that's why it's so helpful. So for example, mutation, right? Like changing things in place, it's a source of complexity because Rich Hickey has great talks on the topic, right? Because now you need to think of how that thing is changing over time.

And if you remove the mutability aspect, right, if you make things immutable by default, you remove that whole time question of understanding your system, right? So mutability is a source of complexity, so it needs to be more explicit. It cannot be the default. It cannot be something that you do automatically. And for example, another way this shows up in functional programming for using more strict languages like Haskell is the whole idea that any side effect that you have in your system,

if your system is changing the world around you, like talking to the database or someone, in Haskell, you need to be explicit and use a monad. But here, we don't have monads, right? But we go with this idea, if you want to do something that's changing the world around you, we want you to be explicit and put it somewhere more explicit in your code. And we wanted to put it apart from the code that does not change the world around you. receive data and transform this data instead of mutating it, which means every time you call a function with the same input,

you're going to get the same output. It's easier to understand what is happening with it. The state that it receives, it's always explicit. There is nothing happening behind the scenes. One example to try, people listening, to try to visualize that. For example, when you're writing tasks for object-oriented languages, sometimes we always had to write that task, you know, we had to set up a bunch of our mocks or relationships

or set up the state you need to get to before you can write the task, right? And that's because, you know, it has a bunch of states, a bunch of things related to it that it can mutate, that it depends on implicitly, right? And that's because, you know, it has a bunch of states, a bunch of things related to it that it can mutate, that it depends on implicitly, right? It's like inside the object. But here we don't have that, right? Like you have functions and everything, there isn't an object state that is a consequence of calling something, right? Everything that you receive is explicit and you need to return everything explicitly as well. And I think that makes wonder to, you know, make your software more maintainable because now you can look at it and say, hey, I see what this is doing because I

can look at a function and see everything it needs to work on. There are no inner state, right? There are a bunch of relationships that can change and affect the whole system. So those are some of the points for maintainability. I'll just expand on that and maybe bring up a specific point when it comes to functional programming. One thing that you often find is you have this, like you said, you're just transforming data, right?

So you pass the data to this function, then this function, then this function, and they all change it in some way, return a new thing. That's a mutation of that previous form. And so you end up oftentimes passing around the same, what I've called previously just a bag of data, and doing things to it. And so my question is always like, well, why don't you just make that bag of data smarter and then it's an object and now it has its own internal things going on and you get back to object-oriented programming. One thing that Elixir does, which I think is really cool when it comes to passing that

same thing into as the first argument to all these different functions that are going to mutate it, is you've introduced a way of just alleviating that little syntax pain, which is your pipeline operator, which is kind of like the pipe symbol and the greater than symbol. Can you explain us? I think that's one of the, if we talk about just language features, I think that's a big one. And maybe it's just syntax sugar, maybe not. But can you talk about the pipeline operator and what it does and then why it seems to resonate so well with so many programmers?

Right. So the whole idea of the pipeline operator is actually to help you and to express like the whole, like I have the data and I want it to go for this step, this step, this step, like the pipe in your terminal. Right. And I added it, I think it was, I added it because I saw it in F sharp or ML or something like that. But who really took it like to the next level was Dave Thomas. So if you buy like programming, like a clear book, the pipeline operators right down in the cover. And he was the one who saw like, Hey, this is the thing that's going to make Alexir click to a lot of people. But yeah, the idea is very simple. And Alexir kind of said,

okay, the subject is always going to be the first argument, which makes everything easier to pipe. And yeah, and that's it. And I agree with you that it kind of gives the idea, well, this is kind of object-oriented, right? Because I could think of, I have like this bag of data and then I could be calling functions in it and the pipeline is going to resemble that. And that's why I think it resonates with a lot of people. But there is one very big importance difference,

which is if you have, like, if we made it objects and the thing about object is that you are putting the data with the things that act on it together, right? You are mixing data plus code. Right. Yes. And one of the things that I end up realizing later is that it brings a bunch of awkwardness into our software. And then it kind of becomes a problem and we end up trying to solve it and ends up creating more problems in object-oriented languages. I hope that I do not sound like a snob saying those things. I don't think so. Truth is truth.

Yeah, it's just like things that I built with time, right? Like those perceptions. Because, for example, after you put those things together, right, you say, okay, now if I want to add something new that works on this data, I want to couple it together as well. So imagine that you have like your object that has a couple of methods that work on this data. We just couple them. And then you have an idea for a new method that you want to add to it. If you call it differently, as different from the other methods, if you cannot simply say object.foo, if you cannot say object my new method, it's awkward, right? So we want to put it in there with the rest of the things.

And for example, Java does not allow you to do it. After you define, you cannot extend it. And then people are like, oh, now I need to sub the case and so on. And Ruby is much more flexible on these matters that we can extend things later, but it generates all kind of weird coupling now, right? Because I have now external code that may be monkey patching an existing class and that adds issues. And that's all because we try to couple those two things in the first place, right?

In Elixir, when you're using the pipeline, there isn't this awkwardness because we are saying I have my data and I'm going to call the function bar from the module full. And then if you want to add your own module with your own functions, you just call it next in the pipeline, right? I'm going to call my new function in this other module, right? And it's natural because that's how you're calling everything. And you can swap easily, you know, change things, you can compose, you can replace the function calls, and you never feel that need to couple it together with the data,

which to me, it's the big win, right? It also has the big win that, for example, if you have an object, you're saying object.foo, where is that foo defined? For example, in Ruby, where I can define things everywhere. It's really hard, right? And you can even think it's defined somewhere, but some other file kind of replaced it by something else. But here you have modules and after the module is compiled, it's done. It's a sealed deal. And I know it's in that place. And if I'm going to look at there, it's going to be there for sure. So I'm going to move

somewhere else under my feet. So I think those things, they matter a lot and it's going to help us write more maintainable code. Let's talk about another aspect which you bring up around productivity and you say it's because the tooling is good. Recently, we've had a lot of fans of Elm talking to us, especially after our show recently with Richard Feldman. One thing he said on that show

which was that the most exciting thing about Elm to him, which again that's the best of functional programming in their browser as their pitch, is that the compiler itself is a huge benefit of using Elm. to be useful and not just segfault or just throw a syntax error and not give you any information about what's going on is huge for productivity. And that's something that Elixir seems to support. As I said, I've been doing a little bit of Elixir

and I'll find that sometimes it tells me not just what went wrong, but gives me a code snippet on like, if you would just replace your code with this code, everything would be all right. That seems like a pretty big feature. Was that a focus for the language early on? And can you talk about why that's such an important aspect of why Elixir is the way it is?

Yeah, that's a great question. So I always said, if you see a bad error message in Elixir, you should open up a bug report. If you get an error and say, I don't know how to fix this. I don't know what is the next step. I don't know what is wrong with my code.

Open up a bug report and you're going to try to make it better. And I don't think our compiler is as good as the Elm's compiler in terms of like telling you what to do next. Evan did really a great job with Elm and having the static type system in that case helps a lot to provide the proper information.

But it's something that we really try to do, right? Like, hey, you know, I don't want to make you clueless, right? Something's wrong. I'm going to try to tell you as much as possible why that happened. And it was there since the beginning because what happened is that I want people to use Elixir, right? And if they're using Elixir, they need to learn a lot of things, right? They need to learn about pattern matching,

as we're talking about, you know, immutability. You no longer have objects and things are immutable. So you need to think in terms of immutability. There is recursion. There are a lot of things that you need to learn, right? So those small things that can get in your way, the silly things, right? They are not the important ones. Like when I do a typo or when I do a small mistake, we need to get them out of our way. Or if they happen, I need to tell you, you know,

how to solve that problem and how to think in the proper way. So that's the reasoning for this. And it was kind of always there. It was something that we always were to, you know, let's make this learning process easy so you can focus on the things that matter and not on those small details, on those hiccups that everyone has, right? When they are learning how to program on a new language. I don't remember who said this, but they said that their first Elixir code, they would just type whatever they think what it should be. And then the compiler would tell them, hey, that's not the proper thing. And they would fix it.

And, you know, they got it working only with those steps. and he didn't have to search on the internet, go and stack overflow, because the compiler, every time he did something that was not the thing that was supposed to be done, it was able to guide him towards the proper direction, which is really nice. Yeah, absolutely. Getting stuck on those little things is always a burden and can really turn you off to a language

when you just can't get it to work. So having it hold your hand as much as possible is huge. I think another aspect, we talk about features. A huge feature in any language is the community, the ecosystem around it. We've been mentioning offhand Phoenix, which is the web framework.

There's also Ecto, which is a database layer. And other projects that aren't Elixir proper, but they are things that you appear to be personally invested in with regards to time and effort and decision-making and stuff. When we had Chris McCord on back in episode 147, I asked him about you, and I was kind of wondering aloud. And I said, I wonder if the reason

why you've been so highly involved specifically in Phoenix at that time is because you see it as a chance to give Elixir a greater success. So if it has a killer web framework, Rails put Ruby on the map in America, at least, and made it something that was more mainstream. And so maybe if Elixir had a Rails-esque type of a project

or let's just say a really viable web framework that people could build web apps with, that would increase the chance of success of the language. I asked Chris if that's why you did it, and he said, well, I can't really answer that. I can't tell you why Jose does what he does. But you might ask him sometime. So I say all that to say this. Here we are. I can finally ask you. You've been highly invested in the web framework and in these web tools. And I just wonder the reasoning behind that. And if I was right on in the sense of,

you see this as a path of giving Elixir its greatest chance for success. Right. So I try to not vinculate whatever I'm doing with the chance of success directly. I obviously want it to succeed, right? But I try not to do things directly because of that, right? So, for example, for the web in particular case,

I said at the beginning about platform attack, right? That we are a consultancy and then we are working with projects and most of our projects are web projects. So, back in 2012, when I asked them to fund, the idea was that eventually we would have Alexi 1.0 and eventually we would have the good web tooling so we can use it for ourselves. The company could use it and can start using Alexi

in production and use with our clients, which is one of our goals. So that was pretty much the idea. So when I was working with Elixir, I had like, I don't want this to be a web-centric language. So I'm not putting like things that make sense only for the web here. That's not what I want. But after we got Elixir 1 out of the way, I said, okay, I'm going to do step two, right? Like this is the second milestone, which is to focus on the web tool.

And that's why I started to work more with Phoenix and work more with Vecto and so on. And other than that, for example, before Alex Rondon came out, I kind of built my own web framework that was not supposed to be used for anything, but just to give an idea like how, where the language was and what could I do with it and what could be improved. So that's something that happened, but I always try to keep this stuff apart. It was never the focus.

So the main reason was really, you know, if we want to use, if Tata Formatech want to use this with clients, we need to have a very good web story. And that was actually the goal for last year and uh i am like and we i think we achieved it with with a lot of success because uh phoenix will know came out and it's not me right it's like the nice thing about phoenix

is actually that i didn't have to solve that problem chris did like of the work. I'm just helping. And this is great. And then I think we're able to get really far with Phoenix 1.0, Acton 1.0 came out. Now the community is starting to grow. We can see more and more companies using Phoenix. And I'm in the last steps. I'm working on Acton 2 right now. That's going to improve based on some of the feedback. I'm also working with Chris and Bruce State on the Programming Phoenix book, which is also going to help with all those things, right? But the main reason was exactly, you know, it's just something that we need to use and that's what I want to focus on. And yes, I kind of knew that if we have that, it's going to help with adoption.

Because when you have just the language, you don't have, let's say, a tool to use it with. This was actually very interesting because when Phoenix started to become more popular, it started to attract different, let's say, kind of developers that were aimed on different things. Because when they're learning only the language, your focus is, oh, I want to learn the language and I want to master it. And when it comes to Phoenix, now we start to see more developers that are like,

hey, I want to build something. And of course, learning the language is part of the process of building something, but it's not my goal on its own, right, per se, which is completely fine, but it's just interesting how the dynamics change. So, yeah, so that's kind of the reason. I think I maybe possibly answered your question.

No, you did. I find it somewhat interesting that you say you don't necessarily like to think about it in terms of what you do directly leading or increasing the chances of Elixir's success, I would expect it to be the exact opposite. You would want all your efforts to specifically try to advance the chance of Elixir's success. Maybe it's just not that deliberate. But that led me to this thought is, what does success look like? I'll tee this up. We need to

take our final break. And this will give you a chance, Jose, to think about the answer. If Elixir is as successful as possible, what does that even mean? What would a success story be five years down the road, 10 years down the road that would make you sit back and say, yes, this was worth it.

This was all that it could possibly have been. What would success look like? So we'll talk about that and then we'll go back into more of the community stuff after that existential question right after this break. We're excited to be working with BMC to spread the word about TruSight Pulse, their SaaS-based monitoring service for cloud and server infrastructure that lets you monitor, visualize, and alert with one second resolution. I had a chance to talk to Mike Moran, the senior architect, about what real-time monitoring is. Take a listen.

Real-time obviously means different things to different people. To us, real-time is one second. So for us, we have one-second metrics on everything that we collect. We'll pull all of that, push it to our servers, and you can see it roughly in about four to eight seconds, depending on where that falls in the interval. So we'll pull one second data, and within eight seconds, you can see it streaming live on your dashboard. So during this conversation with Mike, I was trying to figure out what real-time monitoring means to them. And I was also trying to figure out who might use it and why they would

care about one second resolution timing when it comes to monitoring their infrastructure. And this is how Mike broke it down for me. I think at the beginning, you kind of looked at it and went, that's a very niche set of the market. But I think as things have changed, you can look at e-commerce companies or you can look at anybody who's running an application. We now have stacks that are very nimble and we end up with things like restarts that are quick or our stats change very, very quickly now. So our spikes maybe aren't something that, you know, it's not Black Friday and you end up with this gradual spike or this immediate spike that lasts for a long time.

You now have a lot of things happening because you have so many interconnected systems and you have microservices and dependencies everywhere. Something happening in one obviously affects other things. But if it's something small or happens very quickly, you don't notice that. And at this point with Mike, I was like, well, what's a better example? Give me a real world example that everyone knows about that can really explain how important it is to have one second near real time monitoring on infrastructure level stuff, stuff that really matters, the heartbeat, so to speak, of an infrastructure. And this is what he had to say. It's pretty interesting.

If you're looking at your EKG and you're looking at your heartbeat, how many doctors would ever look at your heartbeat at a minute interval or a 15-second interval? You'd be crazy because you'd miss whatever was happening with your heart, and that's something that you wouldn't want to screw with. Wow, what a great real- world example of what that exactly means. I don't know about you, but I don't want to mess with my heart. My heart keeps me going. Your heart keeps you going.

And if you value the heart of your business, the heart of your infrastructure, you're going to care about one second resolution timing. You're going to care about real time monitoring. And BMC's TrueSight Pulse truly is something you should take a look at. Head to bmc.com slash TrueSight Pulse. All one word, no hyphens, and tell them the change log sent you. All right, Jose, before the break, we teed up a question for you. Kind of a big question.

Look forward and think, what does success look like for elixir what what could be beyond your wildest dreams a success story for elixir if you're five ten years down the road right so um like before the break you said that it was kind of surprising like i that i didn't associate things like to increasing the elixir adoption, right? And the reason for that is because it's not generate any expectations, right? Because if I think, imagine like, I think something is the correct way to do it and that doesn't lead to better adoption

at the end of the day, I should not couple the two, right? In the sense that, you know, what's going to validate if something is good or fair in the proper direction. Adoption is always great, but if the adoption falls because of something, you know, it doesn't imply the cause. And I don't want to associate that in terms of the language building and the whole idea of just not having expectation so your question basically like what I see as a success I I don't have really an answer because I try really really hard to not build any expectation right like to not try to to think about it and just do what I think is right and what I think makes sense,

what's going to make it easier to write maintainable and scalable software that we're talking about and keep the developers productive and so on. So, yeah, so I know that this question in particular, I'm not answering it, but it's kind of on purpose because it's something that I try to stay really far from and try not to get into. But there are a couple of things that I kind of think like, oh, this would be nice, right? So, I mean, in five years, what would really be nice if I am still working on Elixir, right? And there is a community and we are continuing to to grow that's definitely something nice i don't try to think of precise numbers of precise goals but i obviously want that right and um and as i said i want to continue

increase adoption i thought i don't have it as an end goal per se also Also, I want to have a diverse community. So I don't want to be focused like only web, right? Because we have so much with the Erlang virtual machine and the ecosystem that's already there for building concurrent distributed systems. And so I really want to build more and more around that and go beyond web, right? So if you're doing things that are more low level

or building distributed systems, I want to see more of that happening. And we're already seeing there are good talks happening at the Alexio conferences about going more toward the distributed system approach and not necessarily web, which is really interesting. We can learn a bunch of stuff. One of the things that we mentioned about Phoenix.

So in Phoenix 1.2, we are going to have a presence system that allows you to say, hey, someone came up online. And the work being done, it's really nice because it's going to be completely distributed. So we don't need a database to persist your presences. If you have like three nodes and you put a new node up, the nodes are going to talk with each other and it's going to synchronize the data. So we have the whole distributed system working and we have the whole presence system working. If the nodes come down, everyone that's connected to that node is going to come down, that information is going to

go across to the other nodes as well and update the presence status, which is, you know, very interesting ways of doing stuff and Chris and Alexander that are working on that, they are using, you know, like a bunch of research

papers that are from this decade, which are interesting things happening. And I hope it's going to bring more distributed focus folks to, to the language, right. And help build more on that area. And that's, so that's kind of my vision. That would be really nice. I know also there are a lot of folks working on embedded Alex here and starting to hear more and more. There is a fantastic project called the Nerves Project

that's trying to make it really easy to build Embedded software and deploy to your hardware or whatever you have. And I think that's something nice, right? To go and have a diverse community where I can learn more about Embedded, I can learn more about embedded, I can learn more about distributed systems. One of the things I want to explore for future versions is something more related to streaming data, right? So if you have data coming in and you need to process it in

different ways and send it out somewhere, we want to make that as efficient as possible, which I hope is going to bring more data-oriented folks, right? And so on. So that's kind of like my view. It would be really nice to have a really diverse ecosystem and have all the different kinds of things happening in the community. Very good. You mentioned the presence feature, which sounds totally awesome.

I know something similar specifically to Phoenix is coming down the pipelines with regards to presence. And we have a bunch of Phoenix and Ecto-related questions for you, Jose. And as we talked about in the break, we're running short on time here, so we're definitely going to get you back. So listeners, stay tuned for an upcoming show with Jose, and perhaps we can get Chris on as well

to talk specifically about web stuff with regard to Elixir, because I'm all for diversity in that community, and that's an awesome goal for you and one that I hope you achieve. That being said, the web stuff is very exciting to many of us, for sure. But now let's just close with talking about getting started. If you have people out there, they're probably excited about Elixir. Maybe they've

dipped their toe in the water, but they got a false start. Specifically, we had some members in our Slack room talking about getting started from the perspective of somebody who hasn't done functional and is really an object-oriented programmer, historically. And they're wondering what your specific advice is coming from that angle, getting started with Elixir and Phoenix and the whole ecosystem. What are some ways beyond

like, go Google, you know, for things like, what are some really solid ways that people can dip their toe in the language? One option is to go to the website. We have a really, really good getting started guide. And I'm saying that because a lot of people, they tell me that like, I read the getting started guide. It was fantastic. It went for the most important things I need to know and to get started.

So that is a getting started point. And we have already a bunch of resources available. So there is a programming Elixir book we talked about from Pragmatic Programmers, which is going to go through the language and cover a bit of the aspects of building systems. But we also have, let's say, more advanced, in between quotes, books like Erlang, oh sorry, Elixir and OTP in Action or Elixir in Action from Manning, which is an excellent book that's more focused towards the aspect of building systems. And it's trying to put you more into the mindset of building systems in Elixir. So those are very good starting points. And the question about object-oriented, I would say I wouldn't worry

because the majority, for sure the majority of the Elixir ecosystem came exactly from that background, right? So if you're a know, you're going, if you're a somebody, you're going to find resources and or you're going to talk to developers. So we have a Slack room,

if that's more of a thing. I am particularly on IRC. We have an X-Ling channel. So if you're having troubles not understanding some particular mindset, you can, you know, you can go for the resources at a boat, but you can also hop in one of those channels

and ask questions like hey I'm having trouble to let go from some of those particular things or I'm having trouble to express

this particular thing that was that was easy to express in a much more integrated language so what is the correct way to do this here

and we are going to have discussions around it and hopefully you'll be able to get started and have fun. So we obviously have some closing questions to this deep dive.

I got to say, I'm really excited that we've had you on the show. I'm more than thrilled to see your path from beginning programmer to where you're at now with Elixir in this community, this budding website and embedded side and so many opportunities for you. And I'm glad to see that you're authoring a book and going to be at Strange Loop in 2016 to give that talk on the Erlang VM. And one thing our guests love to talk about when they come on the show is kind of not just where they came from, but who out there may have influenced them along the way. So this is a chance for you to kind of mention somebody

that's been a hero to you. So who's a programming hero to you and why? Sorry, I'll have to, there was a misunderstanding because I won't be on Strange Loop 2016. Oh. No, so that's the talk I gave in 2012. It was when I was telling about the the background maybe yeah maybe i didn't express myself properly at the time well we got something some issues then because i found a page on strange lip 2016 elixir modern programming for erlang vm was that in 2012

then that was 2012 okay that's my bad then so i missed i saw it on the on the site and i didn't find it anywhere on their youtube i'm like okay he hasn't given this talk yet then so i thought i heard you say it was in the past but i i wasn't sure we'll leave that in we won't edit that out we'll just we'll let that that blip be there and uh we'll obviously let jose be the correction there for it but nonetheless um besides that who is your programming hero? So that's a hard question, but I will go with Geistu. Because so I have, he did a lot of work on a bunch of different languages, right? From everything you can think of, from like Sch scheme to Java and taking part of, you know,

in things related to JavaScript, Common Lisp, C and so on. So he has been, you know, a great influence on the many languages, right, that the industry uses as a whole. And he has great talks. I like to say like the, my favorite talks of all time is a talk from Guy Steele on building a language. And usually when I'm talking about Elixir, I have a quote from that talk that, you know, a language needs to be a pattern

for building more languages, which is the reason why I always wanted Elixir to be extensible, right? Because our field today is so, you know, so wide that the language needs to be extensible. You need to be able to get it and take it to the domain you are working on. So, yeah, I would definitely go with GuestTool. GuestTool, all right. And aside from your hero, what is on your open source radar?

Obviously, you're writing this language elixir and you're doing so much more around it. But what else out there is on your open source radar? Obviously, you're writing this language Elixir and you're doing so much more around it, but what else out there is on your radar? What's out there in the open source world that's got you excited that if you had a free weekend that maybe it wasn't Elixir-focused or maybe it is, might paint back into the future? You sort of described a bit ago. What's on your open source radar?

So there are two things to that so one is the nerds project i talked about it's part of the the elixir lane community but they are doing a great job with the everything embedded and it's something that i won't say oh i need to to play a bit more and maybe help in any way i can if i can help. So that's definitely something that is in the radar. But regarding the whole, you know, like future of Elixir, I said like streaming data. So I have been following what is happening, you know, with Apache Storm and Apache Spark.

It's things that I am like following on the side, reading about it. There's also the Microsoft Project. It's really interesting. am like falling on the side, reading about it. There's also the Microsoft project. It's really interesting. I've read the papers, but I haven't played with it yet. It's something that I plan to do. What was the Microsoft thing? Microsoft Orleans about virtual actors. So they use that, for example, when deploying Halo.

So the whole idea is that, so today in Elixir, you're kind of like, if you are starting a process that gets a computation, you need to tell exactly, I want to start this process on this particular node, right? So you always need to get, to give that information. And the idea behind Microsoft Orleans, they call it virtual actors. So a good thing that would be like virtual Elixir process. So for example, if you have a Halo game, The idea behind Microsoft Orleans, they call it virtual actors.

So a good thing that would be like virtual extra process. So for example, if you have a hollow game, you would have, and if you just joined in and you want to play on multiplayer, for example, you would send a request to their cluster. And the cluster would start like a process for you, like similar to the extra process, but anywhere in the cluster. You don't say, I want this thing here. I want this thing there. They take care of that for you, which means that if you are not worried about location, for example, if you're having high loads, what you can do is that you can plug more machines and because you don't really care where that particular process is, you can move it around. So they say, okay, I'm going to move this from this machine because this machine is having a spike right now

and cannot handle the load. So it has a bunch of interesting ideas in there regarding virtual actors and process placement in a cluster, which would be nice to explore. Very cool. Well, like I said, it's been a blast having on the show i know uh we've kind of teed up the a tease of a potential i guess take two on this to to get you back on and also chris talks deeper about the website of elixir so if you're listening to this stay tuned to that we

also have a bunch of great shows in the schedule coming up free code camp with quincy larson tiddlywiki with Jeremy Ruffston, and a big one for us. We're excited about the future of WordPress and Calypso with Matt Mullowig. So those are some upcoming shows for us. But Jose, before we close out the show, anything else you want to mention about Elixir,

about anything we've talked about today before we close out the show? I just want to thank you for having me. It was really fun. And if you heard the podcast, go give Elixir a try. We talked about it and try to join the community.

There are a bunch of ways, but if meetups or getting, you know, like with other developers, more of a thing, you set subscribe to the Elixir Raider. Do that because we have a meetup section that is telling all the meetups that are happening with Elixir around the world

and then you can find something close to you that you can go. But there's also a bunch of conferences coming up. We're going to have Elixir Days in Florida. We are going to have Elixir Conference in Europe here in Berlin around May. And the Elixir Days in Florida is March, if I'm not wrong. And we also have an extra conference in the United States about August, September in Florida as well. So, you know, come and be part of the community.

Yeah, a lot of stuff coming up for you then. Yeah, that sounds really good. So we'll make sure we link to a lot of things here. Definitely the newsletter you mentioned, back to the intro to the site, a lot of links in our show notes. So this is episode 194. So if you're listening to this,

go to changelog.com slash 194. Or if you're using a podcast app, check your show notes. There's all the links in there. Don't wreck, don't pull over and try and write it down. We'll just use the show notes for that. But Jose, it was awesome to have you on the show notes for that. But, Jose, it was awesome to have you on the show today. I want to

give a special thanks to you for taking the time to be in the nighttime because we're in different time zones, so you had to really work hard to work the scheduling out with us. We appreciate that. And to those listening, thank you so much for listening. And to the sponsors, Top Tile and Node,

Rollbar, and also Truesight Pulse. Thank you for supporting the show. But, fellas, that's it for this show, so let's say goodbye. Goodbye. Thanks, Jose. Bye. Thank you. We'll see you next time.