The Changelog: Software Development, Open Source - Elixir meets machine learning (Interview)

Episode Date: May 7, 2021

This week Elixir creator José Valim joins Jerod and Practical AI's Daniel Whitenack to discuss Numerical Elixir, his new project that's bringing Elixir into the world of machine learning. We discuss why José chose this as his next direction, the team's layered approach, influences and collaborators on this effort, and their awesome collaborative notebook project that's built on Phoenix LiveView.

This week on The Change Law, Jared is joined by a special host, not me, Daniel Whitenack from our podcast called Practical AI. Check it out at changelaw.com slash practically AI. And back again is José Valin, creator of Elixir, and they're discussing on this effort, and their awesome collaborative notebook project that's built on Phoenix Live View. Of course, huge thanks to our partners Fastly, Linode, and LaunchDarkly. Our bandwidth is provided by Fastly. Learn more at Fastly.com. And we love Linode. They keep it fast and simple. Check them

out at Leno.com slash changelog and get your feature flags powered by LaunchDarkly. Get a demo at LaunchDarkly.com. This episode of the changelog is brought to you by our friends at Influx Data and their upcoming Influx Days EMEA virtual event happening May 18th and 19th. If you've never been to an Influx Days, it's an event focused on the impact of time series data. Find out why time series databases are the fastest growing database segment

providing real-time observability of your solutions. Get practical advice and insight from the engineers and developers behind InfluxDB, the leading time series database. Learn from real world use cases and deep technology presentations from leading companies worldwide. Learn more and register for free at influxdays.com. Again, influxdays.com. all right i'm joined by jose valine creator of elixir and frequent guest on the changelog i think this is your fourth time on the show welcome back back. Thank you. Thanks for having me again. Excited to have you. Lots of interesting stuff going on in your neck of the woods. And I'm also

joined by, hey, that's not Adam. That is Practical AI co-host Daniel Whitenack. What's up? Practical Al. Yeah. Yeah. Practical AI, sometimes it looks like with the font on Zoom, it looks like Practical Al. Yes. So when we record on our podcast, normally I'm known as Practical Al. Well, welcome to the show. I'd have a tool time reference. You'd be my Al Bundy for the show, but that would be too old for most people to get that one. I'll just say you can be

my Adam. I'll be your Chris Benson and we'll co-host this sucker. How about that? That sounds wonderful. I'm excited to be here. Well, I had to call in the big guns because I know very little about this space. In fact, almost everything I know about the world of artificial intelligence, I learned from producing practical AI and by listening to practical Al do his thing each and every week. So that's why Daniel is here. I do know a thing or two about Elixir, but nowhere near as much as Jose. And here we're at the intersection of those two worlds. So kind of an exciting time. And we're here first to talk about NX.

So Jose, what is this NX thing you're here to tell us about? All right. So NX stands for Numerical Elixir. And back in November last year, we started working on this. I can tell more about the story later. But the important thing is that in February, we finally unveiled NX, which is a library, but also this idea of a collection of libraries

to improve elixir so we can start doing machine learning, data science, numerical computing, and so on. So I'll just give an overview of what we have out so far. So everybody's on the same page, and then we'll expand on that. So we started with NX, which is the library itself, is the idea in the library itself. And the main abstraction in NX, as you would expect, is multidimensional tensors. So you can do, when I announced NX, one of the things that I did was that I gave a talk.

And in this talk, I built a MNIST classifier, new network classifier for the MNIST dataset from scratch, just using NX. And so, you know, you can work with multidimensional arrays, tensors. And for those who are not familiar, why multidimensional arrays and tensors? One simple example I like to give is like, for example, if you take an image, if you need to represent that image, if you need a data structure to represent that image, you can represent that with tensor tensor, and it's going to be a three-dimensional tensor, where one of the dimensions is going to be the height, the other is going to be the width, and then the third dimension is for the channels, like RGB and so on. And then if you can represent the data like this, you're going to send this tensor through

data networks, through neural networks, and at the, it's going to tell, hey, is this a dog or a cat, or more complex things. So that's where we started. That was the first building block that we built. And one of the things that people ask a lot is that Elixir is a functional programming language. And functional programming language, they promote a lot of immutability. They promote immutability a lot, which means like if you have a multidimensional tensor, like, you know, if you have a 15 megabytes image and you need to do something with it, you need to transform this image,

each transformation that you do is going to copy the whole image in memory, you know, and do a new copy. So you are locating like 15 megabytes every step along the way. So to solve this, what we did, and this is an idea that we've seen elsewhere. For example, in the Python community, we have JAX. So a lot of the inspirations in NX come from JAX. So the way we solve this in NX is that we have this thing called numerical definitions.

And what numerical definitions are is that they are a subset of Elixir that can compile and is guaranteed to run on the GPU. And that's how we can have numerical computing Elixir, machine learning, and neural networks, because we can effectively look at your Elixir code and say, hey, I'm going to get all of this, compile it to run the GPU, and it's going to be really, really fast. So those are the two building blocks. We can come back to these and talk a lot about those things later. And then we release two bindings for NX.

So one is EXLA. EXLA is a binding for the Google XLA, which stands for Accelerated Linear Algebra. So if I'm using TensorFlow, what is running the things in TensorFlow is Google XLA, which stands for Accelerated Linear Algebra. So if I'm using TensorFlow, what is running the things in TensorFlow is Google XLA. So they're using Google XLA to compile it,

to run on the GPU, to run on the CPU as efficiently as possible. So we have bindings for that. We are also now working on bindings for PyTorch, to be more precise, LibTorch. So PyTorch for Facebook, they have the LibTorch,

which is the C library. We are wrapping that as well. And two months later, so that was in February, we released two other libraries. So one is Exxon. So we started with the building block, which was tensors, multidimensional arrays, numerical definitions. So we released Exxon, which is a high-level library for building neural networks. And we just announced Livebook 2, which is interactive and collaborative code notebooks for Elixir. So that's kind of what we have released in the last two months. And it's just the beginning. There's still a lot of things we want to do, but we're really starting on working on this ecosystem and building it up.

So, Jose, I'm curious from the AI perspective, and I'm going to have to admit for listeners that I know almost nothing about Elixir except what I've learned on the ChangeLog podcast from you in previous episodes. So I'm curious, like from the community standpoint, what was really driving your motivation to spend so much time on these things? And I see like, and we can dig into the individual components, but like you're saying that like the main components that I think could make this very functional, it sounds like are there and are being built. But from the community standpoint, were people requesting this? Were people trying to sort of

roll their own sort of neural network stuff in Elixir? From your perspective, what sort of led up to that side of things? Okay, that's a great question. And to give some context, so one of the things like going way, way back, it always started because of like the Erlang virtual machine. The only reason that Elixir as a programming language exists is because of the Erlang virtual machine. And the Erlang virtual machine was built by Ericsson, which is a telecommunication company,

for building concurrent, distributed, and fault-tolerant software. I'm not going to expand on that. You can check Elixir on the website. But all of this stings for my love for the Erlang virtual machine. So when I created Elixir, I was like, I want to have as many people as possible building on this platform because I love it. And I think other people, they are really going to love it and enjoy it too. So I've created Elixir and I've always thought, I think like in terms of like programming languages,

I really think like that Python is a really stellar example of like tackling a bunch of different problems so i always had in mind that you know i want that for elixir and for their lane virtual machine for their lane ecosystem i think we can grow diverse to solve all the different kinds of problems so i come from a web background i was a member of the rails core team almost like a life ago when i started started with Elixir, I had this obvious web background and that was the first, let's say, dimension that Elixir took off with the Phoenix web framework. So people started using Elixir

more and more for the web. Elixir was already a good natural fit for building distributed systems or anything regarding the network due to the Erlang heritage. But it was like, I always want to try to expand this. And the first time I expanded this was back in 2016, we released abstractions for like data pipelines and data ingestion. So if you need to consume like queues and you need to do that very efficiently, we released libraries for that.

And that brought Elixir to a new domain, which was like data processing. And there are some very nice use cases on our website. So for example, how change.org, for example, is using data abstractions that we wrote back then to process, you know, like, because when somebody, if you have a petition that 1 million people signed, you need to send them an update. Now you have to send an email to 1 million people. How are you going to do that? So we start that segment, and then the community need to send them an update. Now you have to send an email to a million people. How are you going to do that? So we start that segment and then the community starts to grow. So people start bringing Elixir

in the early virtual machine for embedded. So there is the NERSC framework. People started bringing that to auto-view streaming. And then there's always the question, like, you know, why not numerical computing? Why not machine learning? So I always had this interest, like, you know,

I feel like it's part of my responsibility, part of my job to try to broaden the domains and the areas of the language. The community is also doing that a lot for a bunch of areas. But, you know, if there's something where I feel like, hey, this is a good opportunity, we can do it,

then why not? Then let's do it. And this always started, just to finish giving more context when preg prog i always had this interest actually like my thesis my master thesis was in test classification by using test classification but that was like 11 years ago so you know like that was deep learning we're not talking about deep learning at the time yet i think everything was still support vector machines

or kind of state of the art. But I never followed back, but I always had this interest. So in October last year, Prague announced a book, which is genetic algorithms for Elixir. And then I was like, hey, apparently there's somebody who knows things

about AI and machine learning in the Elixir community. And he's Sean, Sean Moriarty. I sent him an email and I was like, Hey, I think the platform could be good for us to do everything in machine learning. He said like, I agree. Let's work on it. And we started working on it.

So it was basically, you know, like, it's kind of like, you know, why not? If we can make it happen, let's make it happen. And we will try to figure out how we are going to you know let's build this and then later we will continue working on how to package and how to sell this to people and say like hey what are the benefits of having those two words like joining together and working together so if we stay big picture but we do a bit of a comparison trying to understand exactly your aim here if i was a a happy NumPy slash PyTorch

like that, a Python data scientist kind of a person, are you hoping that maybe someday the NX-based and Elixir-based tooling would draw me over to Elixir? Are there aspects of it that's going to be well positioned better than Python? Or are you more just saying, well, let's bring this area of computing to existing Elixirists and hope to give them more tools? Or are you also thinking from the other direction? Honestly, I never tried to look at it that much ahead. So for me, my goal right now is that, for example, imagine that you're building an application in Elixir and then you need to resort to do something with machine learning or data science. I'm like, oh, I need to go to Python to solve this problem.

If we have a tooling so you don't have to go there and you can stay within the community, I would already consider that a tremendous victory just because that was not an option in the past. So if people are starting to make this choice, I would already be very, very happy. And I would be like, you know, like mission accomplished. Gotcha. And then we'll see, babe steps. Daniel, what tools do you use in your day-to-day work? Yeah, I like the framing of how you just framed it, Jose, because actually

my tool set, my team's tool set, we develop models in Python using TensorFlow and PyTorch. But typically, in terms of the products that we're building or what we're developing, we're developing either API servers or, you know, something. For the most part, we're doing that in Go. So a lot of times what happens is exactly what you were saying. So we're happy writing our API handlers in Go and everything's nice and wonderful. And then we basically just have to call into Python to do an inference potentially. Now, there is new stuff coming onto the scene in the Go community as well to try to support that same sort of workflow where I would love to not do that.

If I was working in Go and I didn't have to call into Python, that would be super cool. And I think that's still developing. So I totally get what you're saying. Like if you're working in Elixir, then it would be great for those developers to not have to do this sort of awkward call into Python for inferencing. It's awkward and always managing that and monitoring it and all of that is sort of dicey. Also, though, I think that there is this sense in the Python community or, well, I'll say the AI community that Python sort of consume this whole world. But I don't think necessarily out of like a particularly one good reason why it should consume that whole world, because it's kind of like all these scientists or like grad students working on computational science and working on AI they're like well all our stuff that our advisor wrote

is in Fortran I don't want to like write Fortran so I'm going to write this Python stuff that wraps around my Fortran and then like people just start writing Python a lot because it's like pretty easy to get into and they so they do all their scripting in that and eventually like the science world just sort of started latching into python and building things there i don't think it's necessarily like the best tools for ai will be built using python actually i think like a lot of my frustrations in life are because of you know working in python and i'm not trying to bash that because it's it great, like you're saying. I think there is an opportunity for both sides of things, I guess is what I'm getting at.

That's interesting to hear that. Jose, one of the things that you did with Elixir, which I appreciated, and I think a lot of people appreciated because you got a lot of people loving and using the language, right? Is you took all these things that influenced you and that you appreciated,

and you brought them together. With the beam, your love for Erlang was the reasoning. But then you went to your language design and you designed the language and you pulled in ideas from Ruby and ideas from Perl and ideas from functional languages. I'm not sure which ones, but you've told this story before and you can probably reiterate all your influences.

And you kind of made this, what I think is a really beautiful language out of it, but it was based on your history, your knowledge, your taste, what you liked. Here you are doing numerical stuff, right? And you're doing data science-y stuff. And I just wonder like, how do you acquire that taste? How do you acquire that knowledge? Do you just know every domain very, very well? Or how do you learn this stuff? I know you said in your, back in school, you were doing some of this stuff, or statistical things, but how have you come up to speed on what would be an awesome way to do numerical Elixir? Yeah, so this time, it has really been Sean and Jacko. So all the part of like, deep learning and how things should work. Sean, he really the

one leading it. But like the main seed that led to this was actually Jackal Cooper. So when I started talking with Sean by email, before we started working together, I sent a tweet. I don't remember, but it was asking about like some references. And then he pointed me to the Jacks library in Python, which a lot of people are taking it like to be the next big library, potentially replace TensorFlow. That's what some people speculate, right? But it's from Google. There's a lot of traction behind it. And then I was reading the docs for JAX. So we were saying, you know, like, hey, you know, like Elixir is a functional programming language. And as a functional programming language,

everything is immutable, so work with multidimensional data would actually be very expensive. But then I'm reading the docs for JAX, which is a Python library, and then they have quotes like, JAX is intended to be used with a functional style of programming. And then they say, unlike NumPy arrays, JAX arrays are always immutable. And then I was like, what is happening here? So it was like this reference, like, hey, it functions, right?

So that's like my spider senses, they were like tingling, like, okay, wait, wait, wait. So there is something here, that's when Sean and I, like we jump with both feet and we're like, okay, there's really something here. And the whole idea in there is because the way that JAx works and the way that numerical definitions in nx works is that when you are doing all the operations in your neural network like hey you know we need to we need to multiply those tensors we need to calculate soft max we need to do the sum when

you're doing all those computations you're actually not doing those computations at the moment. What you're doing is that you're building a computation graph with everything that you want to do in that neural network. And then they get this computation graph. And when you call that function with a particular tensor with certain dimensions, a certain size, it emits highly specialized code for that particular type of tensor, for that particular graph. And that's why everything is functional, because what you're doing is building a graph, you're not doing any computations. And then you compile that around the GPU.

When we saw this idea, it was like, hey, everything can be functional. And, you know, when it started, it was like a bunch of happy accidents, you know, a book being published. So I like to say, like, I really have a thank you, like, for Prague bunch of happy accidents you know a book being published so i like to say like i really have a thank you like for for pragpro because you know if they did not publish this book if somebody read the proposal that sean sent to pragpro say hey we don't need genetic algorithms book for elixir maybe none of these would have started and then somebody pointed us to jack so it was all those things happening and that kind of like gave me a path to for us to explore and come out of this. And I really think, so I said like,

we are going to start working. And as we build the tools, we are going to try to find like what advantages Elixir can have compared to other programming languages, for example. And it turned out that as I kept saying, what I thought would be a negative aspect, which is immutability,

really turned out to be a feature, right? And it's really interesting because there are some pitfalls in JAX, for example. So if you go to the JAX documentation, they have a long list of pitfalls. So there are some pitfalls in the JAX documentation that they do not happen in the Elixir implementation in NX because everything

is immutable. So the way that JAX works is that in Python, they call it the tape pattern. So basically, as you're calling methods in an object, it is requiring all the methods that you call. In Ruby, we know it as method missing. But there are some operations in Python that they cannot be recorded. So for example, if you are setting a property, for example, in the object, or if you pass that object to a conditional, you don't know that that object is being used in a conditional. So JAX cannot record that structure in your code. So they have some pitfalls, like, hey, you have to be careful.

Or if you have a for loop, if you have a for loop in JAX, what it's going to do is that it's going to unroll the loop, and that can lead to very large GPU code. But in NX, everything is immutable, so we don't have those operations in the first place. And because we have macros, I can actually rewrite the if to be a if that runs in the GPU. So this is really cool. So in NX, when you go to the numerical definitions and you look at the code, that code, no pitfalls, is going to run on the GPU. It's going to be sent to the GPU.

It's effectively a subset of Elixir to run on the GPU. So yeah, so, you know, it started with this small tip and then it kind of spread from there. Thank you. LaunchDarkly enables development teams and operation teams to deploy code at any time, even if a feature isn't ready to be released to users. Wrapping code with feature flags gives you the safety to test new features and infrastructure in your production environments without impacting the wrong end users. When you're ready to release more widely, update the flag status and the changes are made instantaneously by the real-time streaming architecture. Eliminate risk, deliver value, get started for free today at LaunchDarkly.com. Again, LaunchDarkly.com. So sitting on top of NX is Axon, which is NX-powered neural networks.

You want to give us the skinny on that tool, Jose? Yeah, so it's pretty much what the name says. It's neural networks built on top of NX. And so Sean is the one. So a lot of those things, Sean is the person behind it. So Axon, EXLA, it's all Sean's work. And what he did for Exxon is that he built all of the building

blocks of a neural network. He built just using functions. They are regular numerical definitions. They are regular numerical definitions. And numerical definitions are regular functions. So he just built a bunch of functions and then you can compose them together to build

the neural networks and so he did like he built all of this it was really funny because i think we can still find it in the repo he created the initial issue which i think had like 100 check boxes which was just like all the the functions that you use like all the initialization functions, optimizers, layers, activations, everything that you have in a neural network that you usually use. So he listed all those, then he implemented most of those, and then he came up with a higher level, he's still inside the excellent higher level API. So you can say, hey, I have a neural network that is going to be this dense layer, and convolutional layer and this activation and this, and I want to train it and you're done. So, you know, the same level of API convenience that you would expect from like

Keras or from PyTorch is there in Exxon, but the building blocks as well. That's what Exxon is about. It's a little bit, you know, out of my reach of my understanding. And it's kind of funny because I can run the basic examples, but I still don't have a GPU. And then if you get a convolutional like neural network, if you're going to train it without a GPU, it's going to take a lot of time. So I cannot run some of the examples, but Sean, he added already a good amount of examples to the repository. So, you know, we have like some very classical data sets that people use in machine learning, like MNIST, CIFAR.

I don't know if I'm pronouncing those correctly, Daniel, but you probably know what I mean. The fashion MNIST and so on. And he has examples of, and then no algorithms like ResNet and this kind of stuff. And there are examples already in the repository and for those things running in Elixir and

compiling and running on the GPU which is very exciting. Don't you have a GitHub sponsors or a donation button man let's get this man a GPU.

Someone's got to get you a GPU. Yeah I know right. Come on. The world would be a better place if Jose Valim owned a GPU. I'm going to put it on record.

Yeah, I was really like in a, just an aside, I was like, I'm going to buy a Linux machine then so I can have the GPU. And then Apple came out and was like, oh, we have TensorFlow running on M1. But they released just like the compiled executables and not the source code. So I'm like, do I buy a new machine

that is going to take space in my house and then three months later, Apple is just going to, the thing is going to be merged into TensorFlow and I'm never going to use it again. So in this way, like, so I'm just like, I'm suffering for like decision paralysis.

I'm like, you know, should I invest on this thing or not? Well, you've come to the right place. This is Daniel's expertise right here this guy he builds these things in his house unfortunately it's all uh crazy right now i i know we we ordered a server and like we had to switch the gpus because of like i don't know if you saw in videos they kind of got mad that everybody was putting uh consumer cards in their enterprise servers and so that all got

switched up, which I understand their business. But yeah, that whole world is crazy right now in terms of actually getting your hands on something as well. Supply shortages and everything. Yeah, yeah. It's insane. Just scrolling through this, I'm pretty excited to try this on my, you know, little workstation with a GPU. I think it's cool that again, I'm coming not from an Elixir standpoint, but I recognize the API. Like it's very Keras, like this high level API that you're talking about where you're, I've got a dense layer, I've got, you know, a dropout layer, whatever it is that like instantly makes sense to me. I feel like I could take this API and create like my model definition fairly easily. And I really like that being a Python user and coming from that outside

world. Like it makes me want to play with this. If it was a totally like different sort of looking API, I think I would have, I would be sort of nervous to dive in. But I also see like, you have your model struct, you have your layers, you have your high level API, and you talk about it like, it's just a Elixir struct. And so serializing it to multiple formats is possible. And we're talking about the model itself. So I don't know a ton about Elixir structs, but this sort of serializing it to multiple formats is really like interesting to me because at least from my perspective what i'm seeing is a lot of sort of push for interoperability in the ai world where like people like publish their model that they

wrote in pytorch on pytorch hub and like then, I'm over here with TensorFlow, but I can pull it down and like convert it using something like Onyx tools or something and use it in TensorFlow. Or maybe there's all sorts of frameworks out there. And I think people are generally realizing it's not going to be one that wins the day. But interoperability is really, really important if we're going to release models and expect people to be able to use them. So I don't know, was that sort of factoring in your mindset as you're thinking about how to represent models in Axon? Yeah, definitely. When Sean was working on it from the design, we were thinking, how can we get an X model, load that into an Elixir data structure, so we can get that and send to the GPU and have that running on the GPU.

And it goes back to what we were talking about a while ago, that I think the first users of this, maybe I'm wrong, and I'll be very, very glad to be wrong, but I think the first users, they're going to be, hey, we have our data scientists that are really super familiar with this tooling in Python that is very productive, very useful for them. And it's harder to convince them to migrate, but hey, we are running Elixir in production. And I just want to bring that model and run directly from Elixir. And I think

that's very important for that use case. So, and I mean, the whole purpose of interoperability. One of the things that I think it's really worth talking about that I think with this idea, so, you know, a lot of people, they think about Elixir, they think about web, but Elixir is also really good thanks to the NERS framework for embedded. And I think there's a lot of potential in this area of, you know, having machine learning, neural networks running on the edge and nerves can help with that. And that can be an interesting application as well. And that requires kind of the same ideas because you're not going to train on the device, right?

So you need to build the model elsewhere and do all the steps and then bring that into the device. So serialization is there. And I think it's a matter of time until a lot of those things, we are working on them as, you know, it's like we also started a machine learning working group in their Lang ecosystem foundation. So people are interested in this. So it's something that we plan to work. But if somebody is really excited about this, so if you're listening to the show, like, hey, you know, I want to try this out. And maybe I can implement like Onyx serialization and you would like to work with us and the PR, it's definitely welcome. We can have a link to the Erlang Ecosystem Foundation, the machine learning working group in the foundation.

So we have a Slack, people can join, can talk to us. And there's a lot of work to be done. And this realization is definitely going to play a big part of it. Yeah. So how long have you both been working on Axon? Because it just seems like there's so much implemented. You were talking about, hey, we need all of these different layers implemented that people know about. Typically, I see libraries maybe that have a new API for machine learning or something. It seems like it takes them so long to sort of add operations and add, you know, support for different layers and such. And I'm wondering, like, what was your thought process and approach to building this in a way that you could come out of the gates, supporting as much

of that as possible? to give you an idea so sean has been working on it on his free time okay and he started working on exon as soon as we announced the next so he has been working it for two months on his free time and it already has a bunch of stuff if you check like the the readme you know it already has the cool i'm not going to be able to say everything but the dense layers dropout convolutional layers a bunch of optimizers like seven eight so he has been able to add those things really really fast and i think one of the reasons for that's because the foundation are just functions we're just building functions on top of functions so it's it's very easy to to compose

and the other thing is also that i think like one of the reasons i'm speculating here to be clear i think maybe one of the reasons why some of those libraries it takes a lot of time for them to implement a layer is because they are implementing everything right they are going maybe like from python all the way down to the C code and implementing or C++ code and implementing that. While for us, it's a very layered approach where Exum just works about NX and NX is the tensor abstraction. And then we have the tensor compiler stuff that compiles for XLA and working at those different layers. When you're working at Exon or in Exon, you are really at a high level.

You're not really worrying about C, C++, any of that. You're just saying, hey, you know, what are the tensor operations that I need to do? And I think that's why he was able to be so productive in building all of those features in this short timeframe. And I think adding new activations, layers, they're relatively straightforward. What I think takes more time and discussion

is when we need to change the topology because that requires to think about how the struct is going to represent that. So for example, if you have a GAN or a recurring neural network, now you have to think like, oh, if it's recurring, now we need to get the data, feed it back inside. So you have to think how you're going to model that.

But it's mostly, it's just at the high level representation. So that's kind of how things have been structured. Yeah, I cloned the repo and his first commit was January 25th of 2021. It's pretty amazing. With a few to follow. Yeah, I cloned the repo and his first commit was January 25th of 2021. It's pretty amazing. With a few to follow. And it was funny because the first commits are like, add some functions, more functions, adding some even more common functions.

So he's just cranking out these functions, like you said. Yeah, so that was in January. Okay. Yeah, so a couple of months. But while working on that, he was still working on XLA and Annex with me. So we started in November. So in November, he was still working on XLA and Annex with me. So we started in November. So in November, it was Sean and I. We were working part-time. So it took us about three months to release Annex and XLA.

And then Sean, he's still working with Annex and XLA. And then he's focused after we announced it in February. It changed to be on Exxon until we announced it. And now we are probably all like kind of going back and forth between projects. So because there's still a bunch of things that we want to build in Exxon, one of the things that I really want to work on is streaming. So Elixir is really good for streaming and I want to have a very good abstraction so we can start streaming data to be inferred into the GPU

so you don't have to load everything into memory. Or, for example, if you have a webcam or a camera that is your embedded device or you're getting from WebRTC or something like that, and you want to send that straight to the GPU and stream it, so we can do all this kind of stuff, interesting stuff that I think we can do. So yeah, we are going to be jumping back and forth on that. I think it speaks to the power of a solid abstraction, too,

and a layered approach when done well, when you get to those higher layers, like you said, unless you have to change the topology, if you're just adding and building on top and not having to drill down through each time, then you can move relatively fast. There's probably also an aspect of this where it seems like Axon's API is trying to be familiar.

And so a lot of times, at least for me, the slow part of software is getting that API figured out and rewriting that API so that it's better. And maybe there's a step up because of all these other projects that have come before that makes it familiar to Daniel and other people who are working in this world. Exactly. That's a very good point. And I think on the Exxon side, one of the inspirations, I think there is a project think AI in Python, which is a functional approach.

Yeah, there's a team in Europe that writes the spaCy library, which is an NLP library. And I think that their main backbone for that is Think. I see. Yeah. So that has been one of the inspirations as well. And I think there's PyLightning or Lightning Torch or something like that. So yeah, that's a very good point. So if you can look at what people are doing and say hey this is what i think it's good this is what i think it's going to fit very nicely at what we do that speeds up the process considerably as well Thank you. for software-driven teams focused on growing their margins. By analyzing cloud services like AWS and Snowflake, Cloud Zero provides real-time cost insights to help you maximize your margins.

Engineering teams can answer questions like, who are my most expensive customers? How much does this specific feature cost our business? And what is the cost or impact of rewriting this application? With cost anomaly alerts via Slack, product-specific data views, and granular engineering context that makes it easy to investigate any cost, CloudZero gives you complete cloud cost intelligence, connect the dots between high-level trends, and individual

lineups. Join companies like Drift, Rapid7, and SeatGeek by going to cloudzero.com slash changelog to get started. Again, cloudzero.com slash changelog. I mean, there's just such diversity in the AI world in terms of the types of models that people are building. But there is a fairly consistent, like if you look at the implementations, whether it's in the AI world in terms of the types of models that people are building. But there is a fairly consistent, like if you look at the implementations, whether it's TensorFlow or PyTorch or, you know, these other frameworks, you can kind of get a pretty quick sense of how they're building their architecture looking into the source code. And I mean, I'm just looking at some of the layers that are

implemented in Axon. And like I said, I think you've done a good job at like, I don't know how to read Elixir. I can sort of get the sense of what what's happening here. And I think that's a testament to like, yeah, like following some of the inspiration, the good inspiration that's already out there in the world. And also, I think it'll be easier for people maybe that do want to jump and, you know, experiment in Elixir from the Python world, and they want to add their own cool layers into Axon. It's going to be a lot easier for them to jump in and do that. I think if they feel like they're not in a total foreign world, they recognize some of these components and all of that. So I definitely think that that's a good call. I know that some of like data science, AI world

kind of operates with a weird set of tooling that includes these things like called notebooks and other things. I know I saw like there's even some functionality related to interactive coding and cells and that sort of thing too, isn't there? Yeah. So there is a separate project. Another person has been working on this project, Jonathan Klosko. When Sean and I started talking, like, hey, you know, we want to build this foundation for machine learning, numerical computing. And then we mapped a bunch of things that we have to do. And there are a bunch of things that we have not started working on yet.

So, for example, we don't have an equivalent to data frames. So that's an open question that has to be solved. We don't have plotting libraries yet. But one of the things that we want to do was this idea of the interactive and collaborative notebook. And to give a bit more context, Daniel, so we have the Phoenix Web Framework in Elixir. And the Phoenix Web Framework, I think two years ago,

launched something called LiveView, which makes it really easy for you to build interactive real-time applications, but on the server. So without having to write JavaScript, which if you're not a JavaScript developer, that can be a plus. And because the logic is on the server, it allows you to do like collaborative,

because if you have multiple people collaborating on the text, right? Like the server is the one that knows where people are, what they should do, how the text should change. So it's really good for building these kinds of applications. The elevator pitch is not correct, but the one line somewhere that it can say is like, react on the server. This way you can think about 365 Vue. And I said, I want to do this.

We want to build this notebook thing as well, which we called Livebook. So that's the Livebook project. And the way it started was very funny. So we have a project called Xdoc, which generates documentation for Elixir. And we're really proud of it. We think that our documentation just looks great,

and it's standardized. All the projects in the community, they generate this documentation with xDoc. It has a bunch of great features. And somebody, some time ago, opened up and said, hey, you know, this project is using jQuery. jQuery is huge. We probably don't need to use jQuery anymore.

So somebody opened up this issue-to-issue tracker. I was like, sure, sounds like a good idea if somebody wants to do it. And then out of nowhere, somebody sends up a request. They didn't ask if they should do it, right? They just sent up a request, replace jQuery by JavaScript.

And I was like, this is great. I reviewed the code. The code was flawless. I reviewed like the best of my power, CCT JavaScript. And then I went to check and I was like, oh, Jonathan, he lives in Krakow, which is where I live. He goes to AGH, which is where my wife studied.

This is very interesting. And it's like, oh, he has like some Phoenix experience. And he's still a student. And I was like, you know what? Maybe he wants to work with us on this Livebook thing. So I sent him an email like, hey, you know, you want to talk? You know, at this time, we had not announced the next yet, but we have announced some benchmarks comparing like code running on the GPU and not on the GPU, which was like 4,000 times faster or something like that. And then I told him like, hey, do you want to work with us?

And then he's like, sure. But, you know, I'm a student. It's like, no problem. You're going to work part time. So he started in January working on Livebook and the idea. And then we started talking to some people. So there was, at about the same time, John, another Jonathan John,

he had released something like a notebook for Elixir as well, a very bare-bones one. So we had some experience from Python. We brought him in like, hey, you know, if you're going to do this, how are you going to do it? What are the benefits? And then we're like, okay. So one of the things that we want to do is that we want to leverage

the fact that, you know, it's very easy to build collaborative and interactive applications in Elixir. So it needs to be collaborative from day one, and it is. So I gave a, there is a video on YouTube of me announcing Livebook, and it's really cool because it shows how Livebook works.

It shows Exxon as well. So there are some good examples. And so like, hey, it needs to be collaborative from day one. And we really want to be interactive because one of the things, so for those who are not familiar with Elixir, like the Elixir runtime, it's very easy to extract a lot of information from the runtime, like what your code is doing. We break our code into lightweight threads of execution so you can inspect each of them.

So we said, okay, we want it to be interactive, not only for people that are working with machine learning and numerical computing, but if you want to get data out of an Elixir system, like a production system, and try to see like, hey, where is my bottleneck? You should be able to do all that. You should be able to interact with a live system as well and interact with your neural network that is training. So this feature is not there yet, but it's part of our vision.

And then I said, well, what do people complain about in notebooks? That's always part of the research, right? So if we go like to Jupiter jupiter uh what people usually complain a lot what don't they complain about what we heard was like well the form that it writes to disk it's not good to diff it's not easy to virtual control right so how we are going to solve that the dependencies are not explicit so and the evaluation order is not clear as well so how we can solve all those things so you know we brought our set of inspirations, we bought the problems

and we started working on how we want to solve this. And then a couple of weeks ago, we announced it, maybe one or two weeks ago, we announced Livebook. Or maybe it was last week. Anyway, it's there. You can really see a four-hour vision is not complete. You can see the important parts in there of like, you know, it's fully reproducible. The evaluation order is clear. Your dependencies need to be explicitly

listed. So everybody who gets a notebook knows exactly what they need to install and the notebook is going to install it for you. John, he created a format called Live Markdown, which is a subset of Markdown that we use for the notebooks, which is really great because now if we change a notebook, we are just changing a Markdown file, which means you can put it on virtual control. People can actually review your changes without having to, you know, spin an instance of that thing and make that work. So for us, it's a step again into this ecosystem and I think there is a bunch of things that we want to explore and try out and really try to be like a very modern

approach to, you know, for interactive and collaborative notebooks. And there are other things like happening in space so there is Jupyter notebooks, there is also Pluto, JL called me from the Julia folks. There is also DeepNote JL, call me from the Julia folks. There's also Deep Note, which is

a software as a service. So we're kind of looking at everything and coming up with our own takes and ideas as well. That's awesome. I'm glad that when you looked at this, you like took that perspective of like, not we need notebooks. People love notebooks, but what's wrong with them? Because I think there have been a lot of, there's notebook kernels for all sorts of different things for Jupiter, but they all suffer from similar issues. And of course, I love Jupiter and it's powerful and people use it with great success.

But I think after people have used it for so long, they've seen these consistent issues. I think the whole managing state thing that you mentioned and the execution flow is probably the top one on my list. So now you're really tempting me to try out. It also seems like you release something cool every week. I don't know how that works. I don't release something cool every week. So I'm feeling really deficient right now. I'm with you. I don't have anything new to release for now. Until next week. Daniel, what you need to do is find some really talented university students

and get them to, you know, inspire them to work on some stuff for you. I guess so, yeah. Yeah, so yeah. So yeah, Jonathan has been excellent into this. And it was like his first LiveView application. So I think it's both a testament to Jonathan and to LiveView, the fact that he could build this thing

in three months while still studying, working part-time. And go check it out. Go check the video, I think. I'm really excited about LiveBook. It's really interesting. And so, for example, we just merged like auto-completion.

So when you're writing code, there is now auto-completion as you would get from VS Code, where you use the Monaco editor. And everything's like collaborative, right? Like if we have multiple people working on it, the changes are broadcast. And based on this idea that it's built on LiveView,

where you don't have to write JavaScript, like the whole thing, including all the Monaco extensions that we had to do, so it had like Galaxy Relaxer and so on. It's like 2000 lines of JavaScript. That's it for everything that it does work. The whole thing about the notebook is that in my opinion, it was a very different approach to how we approach like NX and Exxon.

It's like, hey, you know, like for NX and Exxon, we're like, okay, let's build this and see where this leads us. But for notebook, it was like, this is an area that Elixir is really good at. And I really want to have our take on this. I think we can make this ours, like our version of this, how our vision, our understanding of this. And of course, that requires looking around. But it was a very different thought process.

Just like, hey, I think we can build this. And I think we can build this great because we have great tools for that. And just to make it clear, out of the box, it works distributed as well. So for example, if you have a bunch of people using notebooks for some reason, and you want to start like five machines in production and have people connect to those machines from anywhere they want, it just works out of the box. There's no need for external dependencies. You don't need to bring grads. You don't need to bring a database. So everything was really built using the, again,

if you go to the beginning of the talk, we were talking about their virtual machine, right? And they're building telecommunication systems, right? Imagine you have this platform and you can build collaborative notebooks, right? So that was kind of our idea, our take. How does it do that? Because it looks like it only runs on local host.

Maybe there's a way to... How do you tell it, hey, I've got 10 nodes that I want you to run across? Is that just configuring Phoenix? So by default, we run it on local host. By default, if we're running our machine, you don't want to expose that and have somebody access the notebook. Yeah, it's like an eval. It's, if we're running our machine, you don't want to expose that and have somebody access

the notebook. Yeah, it's like an eval. It's like a public-facing eval, right? Yes, right? Imagine if you're at an Elixir conf, somebody would just be, who is running notebooks here that I can't... Right now, I think we just need to tweak the configuration file. But one of the things that we are working on,

we are going to get the release, we're going to ship both Docker images and a command line executable. Then we'll have flags for all this kind of stuff, you know, like, hey, do this. And most likely what people want to do is that they want to say, hey, you know, I am deploying this to Kubernetes. So I'm going to use something that uses the Kubernetes DNS manager to connect the nodes. So in Elixir, you would use something like Purage or LibCluster that figure out the topology that connects everything for you. Yeah. And I can definitely confirm that people will want to spin these things up everywhere. Now, I'm not surprised when I hear this, but the first time

I started hearing production notebooks and I was like, how do you have a production notebook? It's a notebook. Like, how are you running a notebook in production? But this is like so pervasive. People are like, oh, this is my production notebook. And this is my, you know, dev notebook and all of these things. I don't know if I go that far because I'm like, I don't know how to support a notebook and production, but it is such a pervasive idea. It's cool to see that as a piece of this. And of course, there's other things too, like you were mentioning, you know, pandas and other things. So for people that aren't familiar in Python, there's a library called pandas, which deals with tabular data and you can do all sorts of cool like data munging stuff so yeah it's cool to hear you say that those things are on your mind and because you

release a cool thing every week you know maybe that will be next week or the following one yeah right now like i think we are going to tackle graphing because graphs because it's part of the notebooks but i'm hoping for like the the data frame stuff other people are going to step in to tackle graphing graphs because it's part of the notebooks, but I'm hoping for like the data frame stuff. Other people are going to step in and we are having a bunch of related discussions on the Arlena Ecosystem Foundation, machine learning working group, and this kind of stuff.

If you want to talk about like, and there is, and sure, like machine learning, right? And then we can talk about neural networks and there's like so much work to be done and so many things to explore. So people that are excited, like jumping and you're going to have a feast. Because we didn't talk about clustering, forests, and classifiers, regressions. And then we can talk about linear algebra.

There is just so many things in the ecosystem that one can build and explore that there is a lot of work to do. And we hope like people don't get more and more excited and they are going to join us in this journey. Yeah, it seems like if you've got the graphing thing going and you're talking about Elixir having this sort of natural abilities with web development, with Livebook and other things here. You know, a big thing in the AI world is monitoring your training runs with a bunch of cool graphs with, you know, something like a tensor board or something like that. So it seems like, yeah, there's like, that would enable a lot of things. It'd be pretty sweet to have your, you know, your training run going in Axon. You kick it off from a Livebook and then you can pull up a, you know, to have your training run going in Axon,

you kick it off from a Livebook, and then you can pull up an interface to see all your nice training plots and all those things. And that's all happening in a really nice, unified, robust way. Yeah, that's definitely something that we'll explore at some point. Probably TensorFlow integration as well. It's something that we are explore at some point. Probably test-reward integration as well. It's something that we are bound to have. Yeah, it seems like Livebook really could be your marketing machine.

It could be like your way in for all the disillusioned notebook sharers out there who've had, like Daniel said, they can do a lot of stuff with Jupyter notebooks or existing tooling, but there's pain points with collaboration, with all these things. I mean, the fact that one of your headlines is sequential evaluation, to me, that seems

like, shouldn't that be how everything works? It says code cells run in a specific order, guaranteeing future users of the same line. Not so quick, Jared. I'm like, that's a feature? Like how things work? I mean, it's kind of the wonderful thing about Jupyter notebooks and the really hard thing about them because like it's similar. Like if you go back in history, I don't know if any either of you ever used Mathematica,

but it's a similar idea. Like you have these cells of execution. It's really wonderful for like experimentation, right? Because you can, oh, you did this, but when you're in experimentation, you expect things to fail almost all the time, right? So you don't expect to have like a script that runs and you unit tested and blah, blah, blah. You expect to try something and fail and fail over and over and over until you like tweak it enough to where it works. And so that's great in the notebook environment if you

can tweak things like that the problem is then like oh what were the four million things that i did tweak to get this to go and what state is saved like in my notebook like i could get it to work and then reboot it and run it from top to bottom and it's not going to work again right so it's it's the good thing and the bad thing yeah and i'm pretty sure it's like this feature let's say the sequential evaluation is going to be a limitation at some point people will be like hey i started training my neural network but now i want to do something else in the same notebook while the neural network's training how can i do that so we'll have to come up with ways of branching, but we want to be very explicit on the model.

So we'll say, hey, you can branch here, or we have been calling it internally because everything is organized in sections. We have been thinking, maybe I can set up some asides. So asides, they fork from a particular point, they branch from a particular point, they branch from a particular point and execute the code based on those bindings. So it's basically the state of the notebook

from that moment on without ignoring the other side. So it's something we'll have to tackle. And if you look at the issue tracker, there are a bunch of things that we have been thinking about. So for example, one of the things that I want to do, so we have the idea, so when you persist in notebooks, you're persisting to the file system. So on the issues like so for example one of the things that i want to do so we have the idea so when you persist the notebooks you're persisting to the file system so on the issues like for

example pluggable file systems and i want to make github a file system so you know you can easily like persist your notebooks to to github and that works transparently from from live book without you having to say hey i need to clone and stuff like that. We can work directly on the repo story. And I think that's going to be a boom for collaboration as well, or not collaboration. I mean, a different kind of collaboration, right? You put on GitHub so somebody can fork and play with it. I know there's like this thing

in the Python world called Binder. So essentially you could create a GitHub repo with a notebook and then you click on the little badge and it just pops up a hosted version of that notebook that that will run. So you can like give it a Docker image or something with all the dependencies. For someone like me, if there was like that tie in with GitHub, and I could just launch a notebook and try like Axon, that's like, I feel like people would just latch onto that so quickly. Then the barrier is not like, oh, like Elixir's sort of new to me as a Python person. So I need to figure out the

tool chain. But really what I want to do is I just want to like quick shift enter through a few cells and see how it works. And that's, that's very powerful. Yeah, that's a very good point. Something for us to look into. Yeah. Well, you guys have done a lot, but there's a lot left to do. What's the best place to get involved? Like you said, fertile ground. What do you say? Hop in and have a feast or something. If you're interested in the space and an elixir,

it sounds like there's lots of ways to get involved and to build out a lot of the stuff that's lacking. So is there a discourse forum or is there a Slack? Is there a community around this? Is it just you and the Dashbit folks working on it? What's the situation there? Everything, we have the Elixir, Dash and X organization on GitHub. But a lot of the discussion is happening in the Erlang Ecosystem Foundation.

We have the machine learning working group. So if you go to the EEF website, you can get all the working groups there. You're going to find machine learning. And then you can create an account. It's free. And then you can join the Slack and we'll be there. So that's where we are usually chatting things. Originally, a lot of those things, they were kept confidential, like live book. But now everything, at least everything that Dashbit was working on, it's out in the public. We don't have anything, no more secret projects. So that's the place to go go where we're talking about things.

We have a monthly meeting where we meet and discuss and exchange ideas. So that's definitely the place. Is NX bringing machine learning tools to Erlang? Or are there other Erlang but not Elixir efforts in this space? You understand what I'm saying? Yeah. Is this the first time in erlang the beam-based tooling around numerical computation is happening or is it like erlang only

things that have been going on i think it's the first time for the the ecosystem and yeah and because you can call you know elixir from erlang with no performance costs whatsoever yeah it's pretty cool right you can just call. Like the numerical definitions, they don't work in Erlang because they translate the Elixir AST. Not the Elixir AST, but they translate the Elixir execution to the GPU. That wouldn't work with Erlang,

but everything that we are building on top, like Exon, because it's just building on top of the abstraction. So somebody could go, get Exon, call it from Erlang, build a neural network from Erlang, and just run it. And it should just work. That's cool.

Daniel, anything else from your side of the fence you want to ask Jose about before we let him go? I'm just super excited about this. Hopefully there is some crossover from the Python world. It seems to me like the timing is such that people in the AI world very much are more open to trying things outside of the Python ecosystem than they once were. And so, yeah, that's my hope. And I definitely want to play around with this and appreciate your hard work on this. And I'm excited to try it out and also share it with our practical AI community. Awesome. And I'm really glad that you are having me on the show

and I was able to share all of those ideas and this work that we have been doing. Oh, you're welcome back anytime. All the links to all the things are in your show notes. So if you want to check out Jose's live book demo on YouTube, we got the link to that. We'll hook you up with a link to the Erlang Ecosystem Foundation

if you want to get involved. Of course, Axon and NX are linked up as well. So that's all. Thanks, everybody, for joining us, and we'll talk to you again next time. That's it for this episode of The Change Law. Thanks for tuning in. If you aren't subscribed yet to our weekly newsletter, you are missing out on what's moving and shaking in software and why it's important. It's 100% free.

Fight your FOMO at changelog.com slash weekly. Huge thanks to our partners, Linode, Fastly, and LaunchDarkly. When we need music, we summon the beat freak, Breakmaster Cylinder. Huge thanks to Breakmaster for all their awesome work. And last but not least, subscribe to our master feed at changelog.com slash master. Get all our podcasts in a single feed. That's it for this week. We'll see you next week. Thank you.