The Changelog: Software Development, Open Source - Running functions anywhere with OpenFaaS (Interview)

Episode Date: April 25, 2019

We're talking with Alex Ellis, the founder of OpenFaaS — serverless functions made simple for Docker and Kubernetes. We talked about the backstory and details of OpenFaaS, “the curious case of serverless on Kubernetes,” the landscape of open source serverless platforms, how Alex is leading and building this community, getting involved, and maintainership vs leadership.

Bandwidth for ChangeLog is provided by Fastly. Learn more at Fastly.com. We move fast and fix things here at ChangeLog because of Rollbar. Check them out at Rollbar.com. And we're hosted on Linode cloud servers. Head to Linode.com slash ChangeLog. This episode is brought to you by Linode, our cloud server of choice. And we're excited to share the recent launch dedicated CPU instances.

If you have build boxes, CI, CD, video encoding, machine learning, game servers, databases, data mining, or application servers that need to be full duty, 100% CPU all day, every day, then check out Linode's dedicated CPU instances. These instances are fully dedicated and shared with no one else so there's no cpu steal or competing for these resources with other linodes pricing is very competitive and starts out at 30 bucks a month learn more and get started at lino.com slash changelog again lino.com slash changelog all right welcome back, everyone. This is the ChangeLog, a podcast featuring the hackers, the leaders, and the innovators of software development.

I'm Adam Stachowiak, Editor-in-Chief here at ChangeLog. On today's show, we're talking with Alex Ellis, the founder of OpenFaz, serverless functions made simple for Docker and Kubernetes. We talked about the backstory and the details of OpenFaz, the curious case of serverless on Kubernetes, the landscape of open source serverless platforms, how Alex is leading and building this community, getting involved, and maintainership versus leadership. So we're talking about OpenFaz functions as a service, a thing that's out there to make serverless functions simple. Alex, open up, tell us about OpenFaz, really just start with the start and tell us where this project came from. Yeah, so when I think about where OpenFaz started and why I even started it, it's such a long, it feels like such a long time ago now in the container world. We've seen many projects come and go.

And one of those projects in particular is Docker Swarm. It's perhaps not quite as popular as it once was. And Docker is now going to market with its own version of Kubernetes on Docker Enterprise Edition. But if you wind the clock back to 2016, I was a Docker captain, which is part of the influencer program that they set up in the community. And I was trying to find new ways of using distributed computing with Docker. And at the same time, I was playing around with AWS and the Lambda service. I wanted to program my Alexa. And what I realized was that I was quite locked into this Lambda developer experience.

And also, I couldn't run it on my own machine, right? I couldn't use the infrastructure that I had or the technology that I was studying and interested in. And so I started to kind of prototype and find a way to run those Alexa skills on my own hardware. And at the same time, that kind of thought experiment led me to DockerCon in May 2017. And there I presented this cool hack to 5000 people. And it turned out that they, they really loved the idea of it. And it captured the imagination. And things were pretty early back then. And the project was just called Faz and I hadn't got a community around it. And I did a call to action at the end of the talk to come and help on the project. And basically nobody, almost nobody responded to it.

And I learned how to build a community and how to promote content and how to kind of rally people around an idea. And when you look back from now, it's kind of amazing journey that I've been on with it. Yeah, global community over 190 contributors. Adam, it's kind of funny, mentioning December 2016, May 2017. I was thinking back to our coverage of serverless on the changelog and on JS party. And I believe on the changelog and on jsparty and i believe on the changelog it started back in i think it was early or mid maybe summer 2017 with pam selly yeah we sat down with her at oscon and i think that show was called the serverless revolution or something but even on that

show see she said it should be called functions as a service right that was one of her kind of her rallying calls is it shouldn't be called serverless. It's a misnomer, it's a problem, and we all, I mean, the name serverless has gone through the ringer, but the hype cycle, we're still on it. But it's interesting that at that time, Alex, you were out there building a thing called FAS, and eventually OpenFAS. Yeah.

So you were ahead of the game there. Let's talk about serverless itself before we get specifically into OpenFAS and what it offers. Because it's been a thing, like I said, we've been talking about it for a few years now. I'm sure we weren't right there at the bleeding edge, but as you probably were in December of 2016, a lot of people were trying out Lambda. There's been people that moved their business over to it. There's been people that have tried and had false starts.

As you said, there's lock-in problems. There's visibility and tracing problems. There's lots of unsolved problems or problems that are being solved. And in a recent talk you gave, I think it was maybe a year or two back at GoTo, you talk about the serverless hype cycle. And at the time of that talk, it was kind of on the upswing of excitement. There's a point where the hype reaches its peak, and we weren't there yet with serverless as an idea and a construct. Where are we at now in your estimation and where are we headed? Yeah, I mean, this is what I mean is that in the container and tech world, we're in

a bit of a, like a time warp. That was only a few months ago in October and yet it could have easily been two years ago and still have applied just as well. The hype cycle is a completely normal part of our industry. And just like the seasons, the hype cycle will continue and it will be the next technology and the next technology. One of the things that I really like is I saw a graphic on Twitter by a guy named Seb and it's a picture of a guy shouting service smash over and over again. And then there's some people underneath him saying he's broken, not this again, put him

with the other ones. And they're all shouting infrastructures code, blockchain, serverless. Right. And I think it just reflects how as engineers, we do like people to be honest and say things how they are. And we want to know that technology can solve real problems. And I think when the next thing comes along and it's clearly overhyped, hyper marketed, but doesn't have a substance to it, we get frustrated as a community. But my intent in that talk was to kind of tell people that this is normal and it's OK and it's going to get worse but the promise of the hype cycle is once we get to peak expectations

we'll then get disillusioned with this and then finally figure out what it is useful for and some people are on that journey already but i think the hype cycle is really referring to the industry as a whole rather than you know early. Yeah, so that's funny. I'm now looking back at the video. It was just last October, published in January. I'm not sure why I thought it was so far back, but like you said, things move so fast. That's actually one of our oldest sayings is open source moves fast, keep up.

So we are well aware of just the fast-paced moving of our industry. And it seems like in the operations space, the Kubernetes world specifically, things have been moving so fast and there's been a cambrian of explosion of different offerings uh we've had the cncf on the show laying out their their landscape for cloud natives and to me honestly sometimes it looks a little bit like a minefield or a i don't know a quagmire and i've often asked people like when's it going to just shake out and we're going to have some winners declared so we can just use the technology and benefit from it. I think we've seen

that with orchestration and Kubernetes, but definitely at the serverless platforms, well maybe OpenFast will be the cream that rises to the top of the crop. Interestingly, when I came across this project, it was on a blog post by Abraham Ingersoll, and he had laid out kind of the state of open source serverless platforms. And this, what he said about OpenFaz actually got me to contact you. He said, OpenFaz is utterly fascinating. It's the only contender boasting a license other than Apache 2.0.

Not sure if that's 100% true, but it is MIT. It's extremely community-centric, added Kubernetes support in mid-2017 after originally targeting Docker Swarm, and is deliciously lean. You think that's a fair description of what OpenFaz

is? I did White Quay ofraham on his initial post and one of the things that that i kind of um i'm learning as a as a maintainer and as somebody who's who's really proud of what the community's created is how to influence people that have done their own research and maybe have some some gaps in it. And so we worked very friendly with Abraham. He was a really nice guy and he's written something that's very generous, but I think it's also true. And yeah, it's MIT licensed when most of the other projects are Apache 2. One of the main reasons that corporates pick Apache 2 is because theoretically offers a patent protection on top of MIT.

One of the reasons I picked MIT was that I was trying to figure out what the best open source license was going to be for the project. And I wasn't particularly experienced with it. I'd used MIT a little bit in the past, and it was actually something that I could read and understand very easily. In contrast, Apache 2, I think is much more complicated. And so it went well with the values of the project of making the simple and making it accessible to a community. Well, let's see how simple it is just to describe what it is and what it offers. Maybe from a first run perspective of somebody says, okay, I would like to run some functions as a service,

right? I'd like to execute my functions that I have or haven't written yet. Choose your own adventure on a cloud. And I heard OpenFaz is cool. So what happens next? Tell us the use case and what it feels like to use it. Yeah, so a way that I find, the easy way I find to kind of relate this for people is to kind of look at what experience they may already have. And the experience many people may have is of a cloud function where oftentimes text is typed into a text edit box on a UI. There's no compilation checking or unit tests. There's no good engineering practice there. It's basically hacking and production. That's then deployed to an endpoint somewhere and whatever code you wrote in, if it's supported by

that platform, will then be executing for your end users. Some of those platforms now allow you to upload zip files. But again, there's some potential problems with how dependencies might be managed for some of the code that you need to upload. So with OpenFAS, it takes a slightly different approach and looks at, you know, what's the point at which we can consolidate. And at the time, I believe that to be the Docker container. And I still strongly believe that is the industry standard way of shipping code. And so with OpenFares, you use a CLI to scaffold a function in whatever language you care about or microservice. You can also use a Dockerfile, plain Dockerfile.

And then you'll see your handler just like you would in a cloud function. That will receive a request and then you populate a response. A really nice real-world example, and it might sound contrived, but this is actually a real- world example, is resizing images. There's a startup out of Bangalore called IconScout. And my main contact there just recently wrote a blog post on the openfaz.com about how they're using OpenFaz to resize all the images for theirogue of icons and stock photography. So he found that very simple, that developer experience of knowing exactly what NPM module he needed, how to access it in Node.js, and then it was really just a case of him looking at whatever had been configured for that

deployment, looking at the requests coming in, reading a file and then outputting one at the end. And that's really one at the end. And that's really one of the classic use cases that people often talk about with serverless. Yeah, it seems like the sweet spot is, I don't know if one-off or ad hoc is maybe a better term, ad hoc functions or functionality that you want to have available to you or to another service. Maybe even you think of them as a microservice. And in terms of where it runs and how it runs and all these other things that we usually have to think about, provisioning and scaling and everything else,

it's like, I couldn't be bothered. I just want this to run somewhere when you hit this endpoint. It seems like that's really, really the sweet spot. Yeah, I mean, the way that I often introduce the paradigm is to look at what is serverless anyway. And from my perspective, it's an architectural pattern or an approach to designing systems where there's a decreasing concern for the infrastructure

this stuff is running on, how it's packaged, and also what services you're consuming and where they live. It just becomes a configuration or a string or a connection string. And so microservices tend to have multiple responsibilities that may be quite cohesive. It might be an employee or an enrollment service that maybe creates updates, deletes, and does a few other things, maybe generates a report. In the function world, you may have those as separate endpoints or separate deployments that can scale independently. And you may actually find that they live quite well together as a single function. But really looking at decomposing things down, and those microservices really aren't micro enough

in some cases. And it might actually be that we can get this promise of code reuse that we never really realized as an industry and finally get that through functions, right? So the resize image can now be used by a number of other functions or a number of other microservices. Yeah, I think the real mental hurdle for folks, and I guess maybe I'm just speaking for me, so I won't say for people, but for me with this is what scale of function is appropriate?

And I think the idea of a microservice, as you said there, maybe has too much functionality to be a single function as a service. Maybe it's a couple of functions talking to each other. When I think about functions that I write in my software, you know, sometimes they're five lines of code. And then you have other ones that are tying together other things. And you look at it and you're like, wow, this is like a really big function. And so I wonder, are there heuristics or is there best practices on the size and scope of a function that runs as a service that are being developed or that are out there to be read?

Yeah, so you can get an opinion on everything these days. And there are those. And there have been some Twitter threads that I've watched. And people have said functions must X, Y, and Z. This is a best practice. And then the way I've seen users over the last two and a half years use OpenVAS is they're solving real problems and they're doing it in a way that makes sense to them. And at the end of the day, I'm not bothered whether they have three or four different things that they're doing in a function.

And I'm not going to tell them either. Where I see the core value is in having some principles about what is a stateless and ephemeral workload and then allowing people to solve problems the way that makes sense for them. So generally, a serverless workload or function is short-lived. It's generally single-purpose, has no state between invocations and auto-scales for demand. Now, there are some practical reasons why you may want some state in terms of a connection pool for a database. But the state that I'm talking about there is really things like having particulars in memory. A lot

of monolithic applications would perhaps rely on certain things being established as singletons and knowing that that's there for the next request. In effect, it's taking all of that and it's forcing you to make your code cloud native. And by the way, there's not a lot to really differentiate between a function and microservice. And that's why they're so easy to deploy on Kubernetes. And I wrote a blog post earlier last year about stateless microservices. And the premise is that if you conform to a certain contract, serverless contract, then you can deploy your microservice on OpenVAS, access it through all of the ecosystem, get the metrics and auto scaling for free. That's interesting. So what does that contract look like?

What is all in there that you have conformed to just out of broad strokes? Yeah, so this is a contract that basically I kind of landed upon when I first put the project together and it was packaged in a container in a registry, it listens to HTTP traffic on port 8080 and it has a health check. Furthermore, it needs to be ephemeral such that at any point it could be replaced by identical copy and serve the purpose. That's it. I thought you were going to keep talking. That's a short contract. It's a short contract and it's one that other parts of the industry have actually started adopting.

Google launched a new project last summer called Knative, and I just saw they launched a new managed service called Cloud Run last week. One of the things I was able to do was take an OpenFaz function and deploy it straight to their platform. It's not the same thing as running it on the OpenFaz function and deploy it straight to their platform. It's not the same thing as running it on the OpenFaz platform, but it's the same code and it's deployed on their system, which is basically a container executing system. And that's because they somehow either by luck or by looking at the traction the project had gained, they picked the same contract that I defined and we've been using in the community.

That's nice. Are there any efforts to formalize that into a, make a contract around that particular contract? This is what I'm hoping to do. Part of my talk at GoToChicago coming up at the end of a month is, I have a talk there called Serverless 2.0. And part of my preposition is, well, we have these problems of portability and finding a standard, and we can't write a

function that will run on every proprietary cloud vendors functions framework. So perhaps we need to look at something else that's a bit broader where we can agree. And seeing this in Google's project, I think is a positive sign that perhaps this could actually be where we can start to consolidate around a contract. What all players would have to get involved there? Anybody else running cloud services in order for that to actually be adopted as what you would call a community adoption of that yeah aside from open paths you know i think that um this is one of the things that can be incredibly hard to get to and does need a lot of coordination one of the ways that the serverless the broader service community has collaborated in the past is through a serverless, the broader serverless community has collaborated in the past is

through a serverless working group, which is part of the CNCF. And I've been attending the calls and participating in demos and getting my community involved in that pretty much since I heard about it after my talk at DockerCon. The first output of that was something called Cloud Events. And that got adopted as a sandbox project by the CNCF. And what Cloud Events does is it started to say, well, I have all of these events in, for instance, Azure, when cloud storage is accessed or when a database row is updated, but every cloud platform needs a slightly different way

of consuming and producing the events. And so what we were trying to do in the working group was to come together with something that all of the open source frameworks would be happy to implement, but also that we would be able to see become a standard and have some of the cloud providers adopt as well. I think the first cloud provider that probably adopted it was Azure. And that's because its spec was very close to the one that the work group came up with. And then most of the open source projects that you'll see in the serverless landscape of the CNCF will have some support for This episode is brought to you by GoCD. With native integrations for Kubernetes and a Helm chart to quickly get started,

GoCD is an easy choice for cloud-native teams. With GoCD running on Kubernetes, you define your build workflow and let GoCD provision and scale build infrastructure on the fly for you. GoCD installs as a Kubernetes native application, which allows for ease of operations, easily upgrade and maintain GoCD using Helm, scale your build infrastructure elastically with a new Elastic agent that uses Kubernetes conventions to dynamically scale GoCD agents. GoCD also has first class integration with docker registries easily compose track and visualize deployments on kubernetes learn more and get started at

go cd.org kubernetes again go cd.org kubernetes So Alex, I have done a little bit of serverless on Lambda. It was probably a year or two back. I know they've added things, but it was very much use the tools that they have. Specifically, it was like, well, it's going to be JavaScript. I know they've added languages, but you mentioned containers with Open Fast. So I assume that means if it can run in a container, it can run on Open Fast. So any language, any framework, pretty much anything? Yeah, that's, that's absolutely right. And that's been one of my goals from the beginning, in terms of if you have this experience where you really don't have any kind of infrastructure and you

can't even find it with Lambda and a managed service that may come with an SLA, if you're going to offer an alternative to that, you need to have some unique selling points. You need to have some value add over and above a clone that just happens to work on your own hardware. And so that's one of the reasons why I've never called OpenFaz a Lambda clone. It very much is something else, which is serverless functions made simple. And one of the ways that that's done is through using that container image and having that contract that we talked about so that whatever binary you have, whether it's for Linux or Windows or even for an ARM device, it can be packaged in a

Docker container and then run, built and scaled through the project. That sounds awesome right there. So you even talk about binaries such as FFmpeg or ImageMagick. I mean, there doesn't seem to be, like if it can run in a container, basically, does it have to have some sort of boot up time or anything that runs too slowly maybe does maybe breaks that contract yeah so the timeouts in a proprietary product are very often set in stone and there's a very good and smart reason for that which is because often an sla is offered by that company and they want to make sure they have a great experience for their users however if you're deploying your own functions and you can commit to a longer or shorter

timeout, then you should be able to pop the bonnet and actually do that. And that's part of the freedoms that you get from using something that's open source is that there are same defaults, but you can actually get in there and you can tune them as you need to. That's awesome. And then when you're ready to deploy, is there specific clouds you can use? Is there like an OpenFaz specific cloud? You mentioned it runs on Google's new Cloud Run via that contract. Yeah. So if you like, there's two parts to the story. When I talked about Cloud Run, the first part is packaging up

your code into a container and that container having a contract. And the second part is actually being able to deploy it to a platform. Now, when you're deploying that with OpenFaaS, you can deploy it to any cloud you like, including your local computer, your laptop. The new stack surveyed about 150 developers and one of their top sort of of pain points was and 60% of people said this debugging and tracing is a problem and lack of skills and so for something that actually claims to be no no ops completely easiest can be it's kind of strange that almost 40% of people thought they had a lack of skills and 60% of people had issues with debugging. Now, if you deploy with OpenFAS, the OpenFAS on your laptop is exactly the same as the

OpenFAS that you may deploy to DigitalOcean's Kubernetes service or Amazon's new EKS Kubernetes service. So that's another. That means there's an opportunity for you to open up to that. You know, some may say the cloud native landscape is confusing and overwhelming, but actually it represents a huge thriving ecosystem. And all of the tools that are built to target containers and a cloud native approach will work with tools like OpenFast. And so very often when I've been working with end users like Vision Bank out of Paraguay. We've seen they've

just picked the favorite tools from the cloud native landscape or ones that they're already familiar with. And because of the way OpenFAS is so pluggable, they've been able to build something that suits them really well. So what would be the built-in or the de facto logging, tracing, debugging story if I was just using OpenFaz directly and my functions that I'm here writing on my laptop, what would that look like in terms of me being able to poke at my code and see what it's doing? Yeah, I mean, there's a number of ways that you can do that. One of the things that I think that I've sort of tried more and more in my career to do is to not get into a

breakpoint debugger. With some kind of coding, like writing a front end in React, and just being completely confused, there's definitely a need for that kind of thing. But as much as possible, I've tried with all of the functions I've developed to just write unit tests locally and figure out what I'm doing, and then deploy that into the project. Some of the ways that you can get data back even using a lockdown platform is through simply logging things and the logs are fully accessible because we're just dealing with a container. In terms of deep specific debugging tools there are a lot of different options out there. I recently put a video together with Node.js showing how if you pass the correct node, I think node inspect environmental variable, when you deploy your function, it opens a

second port and you can connect straight to the function from your VS code or IDE and you can hit those breakpoints without actually rebuilding the code. Something that you struggle to do with a lockdown platform. Changing gears a little bit, I noticed on the website you do have what you call pre-built community functions from the function store. I kind of like that idea. Just go to the store, pick your favorite function off the shelf and slap down your credit card and take it home with you.

What are community functions? Is this like, hey, you can resize an image with this community function? Is that the idea? So yeah, the function store came about, it wasn't, you know, like OpenFast has been a journey to start with. It didn't even have a CLI and then we built one and then we made it amazing with lots of input and feedback from the community, and it didn't have a function store. And if you look at almost any successful project that's taken off and had traction, one of the core things that it has is an ability. This is going back to the same thing we saw with Web 2.0, is this idea of sharing and participating and discovering content.

And so I noticed different people were writing very similar functions for things like detecting faces in an image and building the store actually helped them find that. And then I started to put out a call to action and send people t-shirts and things like that to try and get it populated more and more. And so you can find interesting things that you can deploy very simply and start to make sense of functions. One of the things that I quite like is a Tesseract function. Vivek Singh from Bangalore, one of my contributors, he put that one together. And simply because most projects and open source software either works in Node or Python or has instructions for its own Docker file already. It's very, very simple to get from that to something that's

deployed in OpenFaz and then submitting it into the store, something that's now an icon on everybody's screen who's got OpenFaz and they can deploy it in one click. That's super cool. I was curious if there's some sort of a web interface or some sort of way of looking at the store without having OpenFaz yet, because that would be a nice way of bringing people in. They find something that they need and they say, oh, I can run this on OpenFaz. And then they're like, well, maybe I should check out this OpenFaz thing because I would really like to run this function. Is there a way of seeing those outside of the software? That's a really great question. And it's something that people have raised in the past. If anybody's listening and they'd like to help contribute, this is a community project. And that would definitely be something I'd be interested in

getting some help with. Pull requests. Welcome. It sounds like. Right. Maybe that's a good cue to move a little bit into the community, because Alex, one thing you have going for you, as we can tell here, is traction and is a community, as I mentioned earlier, over 190 contributors. And Adam, I know you were curious a little bit about how he's gone about that. And it seems like to a certain degree, maybe it's coming out of some of your foundational principles and precepts for the project. So maybe we start there and just dig into how you've built not just OpenFaz, but you built a community that's building OpenFaz, which is really impressive. Yeah. So when I go back and I think about this project, you know, I think I shared the example at DockerCon that I'd built

this project. People are really interested in it. It received a lot of attention there. And then when I got back home from Austin, there were no pull requests. There wasn't even a dedicated Slack. And so I started to model what I'd seen as a Docker captain in the Docker community, which, as you know, is one that's very outgoing and really embraces people running their own meetups and participating. And I started to model some of the stuff that I'd seen there and try it out for size and also learn some things at the same time. And I gathered together probably around 10 people that had worked with me on my other open source projects. So I just knew one way or another. And we had something like an Alex Ellis Hackers Slack that we renamed to Open Fast Slack, started to have Zoom calls. I pay up, you know, I paid the

$15 a month just out of my pocket money to have a Zoom call that we could all join. And so it'd be seven o'clock on a Sunday night and it would be every week. We just get together and talk about the project and what we wanted to do. And then I really started to think that we needed some values writing down. And it wasn't that these were ones that I just went away and had too many espressos and came back and said, this is what we're doing. It was what we'd already done up to that point. And so I started to write that down and kind of reflect on it. And the three key values were developers first, operationally simple, and community-centric. Now, the community-centric one, you can already see how that was starting to become really applicable.

There were developers from all over the world joining and from the US, different jobs, different backgrounds. Today, there's 197 people that have submitted pull requests and had them merged, which I think is pretty amazing. In terms of the operational simplicity, I had made a contact at Suze, and it was a guy, Troy Tropnik. And he'd been in an executive planning meeting, and somebody there had got excited about OpenFAS in the ranks. And he thought to himself, I'm just going to try it. He was able to have it up and running on his laptop excited about OpenFaz in the ranks and he thought to himself I'm just going to try it. He was able to have it up and running on his laptop in about two minutes and

he was deploying functions and that's while half concentrating. The reason why is because it's operationally simple and that's something that has to be guarded and it's something that has to be defended and it's something that doesn't make sense to you know big enterprise java architects when they go about building a serverless framework they may go about it in a in a very different way because they've got different concerns this is a project that we want people to use and want people to love and we want people to love. And that goes back to that developers first. Especially when you have so many people trying to figure out their path. Adoption is core to open source.

So if you make it difficult to adopt and or get started, which is kind of what you're talking about here, being operationally simple, you're not going about the right way where you can let people get involved sooner and get their feet wet and get some courage i suppose a win under their belt as you've said jared before around things because that's you know as soon as you get a first win even if it's small it gives you enough confidence to kind of keep trudging forward yeah that's that's definitely a part of it also something that's helped is having um guidelines of how new contributors are expected to put that together or any contributor. So the contribution guide is one that, particularly when I was with VMware last year, that other teams there were looking to for inspiration. And that people have often asked me about how, you how do you even create the zoom call how do you start to get people to come and talk about the project and to be present and for me it goes back to this idea of investing in people and investing in individuals it's something

that can be incredibly demanding uh personally and i think i've sacrificed a lot but at the same time it can be really rewarding when i see that one of my friends a core contributor at the time basically was uh i think he was made redundant he was very he was very outskilled and he was able through his work on openfast to go and get a job for double the salary that he'd been on before. And he's now moved on to something even better from that. What do we want back to that Zoom call? Because I'm kind of interested there, but I also want to talk about, you know, kind of diving further in what you mean by developers first and how you say you defend

or how you defend or even define operationally simple. I think you kind of gave an example of that, but what does it really mean to you? So let's Zoom back to the Zoom call because that's kind of interesting. And just the fact that you kind of have this weekly opt-in, come if you want to kind of meet up. Can you describe sort of like how long it was, how you were organized it, how it was self-organized? What did you expect from that? And kind of how has it been repeated since then? Has it been weekly for years now? So I have the, you know, we have pretty

much all the recordings on YouTube on an unlisted playlist that people can get to. But one of the things that I found really inspirational was some of the old videos I found where Steve Jobs was talking to his early company about his work at Apple or creating the subsequent company Next. And he always went back to this example of creating the Apple One. And what he said was that he and his friends wanted to create something so they could use it and so that their friends would want to use it too. I think that was really the core idea and the kernel of what we were doing with OpenFast and why it drove to those values. So some of the early people involved in the project

were coming along for that reason too. I think that's why they put a stake in it and why they're still there now is we were and still are building something that we want to use, enjoy using, and we want our friends to use too. So is this still a weekly thing then happening you mentioned you got it all on youtube is it yeah so i mean still happening every sunday at

seven today i mean it was like weekly by and then every two weeks depending on what what was um available at the time recordings are there the way that i set it up was in the beginning i'd try and come up with a word or a theme and it might be something like contribute or build or engage and try to communicate the values try to rally the community around a certain area or topic or call to action but also i would often give over half of the time to going ram robin around the room not like a scrum call because nobody likes feeling like they're at work when they're not but in a way that actually gave people a chance to say where they're at and what they were doing yeah and some of the best calls that we've had are where I've had like an update from a conference

or a specific message or roll downs and actually most of that has just been parked because we spent 40 minutes engaging with each other and actually seeing where people are at. There was a call that I did about 18 months ago, how to contribute to OpenFaz. It was probably the most oversubscribed call that we'd done. Then I repeated it again about a year later. And again, it was like the biggest call we'd had. Why? Because I think people want to have a sense that they belong to something bigger and that their contribution counts. You mentioned earlier, you said you sacrificed things to build this community. Obviously, when you upgraded to the paid Zoom, you're sacrificing some of your hard

earned cash. What are some other things that you've sacrificed to get this ball rolling personally yeah i can think of the of about half a dozen software maintainers in the industry that i know quite well and we i think we all feel like we're in the same boat which is that being a maintainer and leading a project can be a very thankless task where people will expect much from you and very high standards and they'll just potentially just keep taking until you burn out. And so one of the things that I think people learn, certainly that I learned very early on, is to try and set boundaries. My wife has helped a lot with cooking meals when I still had a day job, doing most of that kind of

stuff so that I could spend those extra two or three hours working through issues or helping users or whatever it may be. It's also meeting with the end users, right? So working late into the evening, meeting with BT or Minio or whoever it would be at the time who had reached out very excited about the project and and kind of had this expectation that a big company was behind it with a lot of money and a lot of resources and actual fact um you know it takes a lot of energy to keep something like that afloat out of flight. errors in minutes and deploy with confidence. Catch your errors in your software before your users do. And if you're not using Rollbar yet or you haven't tried it yet, they want to give you $100 to donate to open source via Open Collective. And all you got to do is go to rollbar.com slash changelog, sign up, integrate Rollbar into your app. And once you do that, they'll give you $100

to donate to open source. Once again, rollbo.com slash changelog. Let's break down the first two pillars of your values. Developers first and operationally simple the example for the second you mentioned spin up time you know to sort of adoption focused but break down for me kind of what you mean by developers first you know what do you what lens do you look at that from how do you ensure that's true yeah so the way that i look at that value and the others is a compass direction and some of the ways that that relates to the project is things should just work. And it should always be as much as possible, only one click away and follow the Unix principles of convention over configuration, which is one of the probably one of the reasons why a lot of people

think Kubernetes is complicated is because it goes the opposite way. It's configuration over convention. Configure everything under the sun and maybe that's right for Kubernetes, but for a platform like OpenFaaS, there's targeting developers and productivity. Some of the ways that we've done that is by creating a CLI that does the Docker build for you and types in all of the correct arguments that can take a handler for Node.js and put all the right things in all the right places so that when you come to use that on a daily basis, there's no duplication between each of your Node.js functions. We've been able to abstract some of that away in a way that makes sense. Now, the CLI is also one of the areas, I think it might still be the second

largest area for commits in the project. It does look like it is on the stats page. And that again is because developers, at least the ones in our community, love CLIs and they love to make it theirs and make sure it makes sense. And so that's why so many commits have come into that area. There's a UI that also is very popular. It's not particularly sophisticated, but it kind of shows you where you are and what's available. And that's where you can interact with the function store as well. And so by getting a script or one command that will deploy everything for you in about 60 seconds,

by doing Ardandis to make sure that the instructions you see are always correct and will always work, and really protecting that and checking it and going over it again and again. I think it's one of the things that means when developers use OpenFAD, the feedback that predominantly get is, we get it, It just works. We know what to do when something goes wrong. We really like it. Can you send us some stickers and that kind of thing. And that, that shows me that we're kind of going along the right

lines with that one. When you add more people, because this is called community centric. When you add more people, the simplicity is somewhat of subjective term, meaning that my version of simple is not exactly the same as Jared's. How do you ensure, you mentioned the word defend earlier and protect, how do you ensure and defend that state there? Is it in pull requests? Does it simply apply to code? Is it sort of project driven or is it sort of community focused, this operational simplicity? Yeah. So one of the ways that I think we get to this is by having these values and seeing what they look like and then working with that. Now, there are a core group of people called the core contributors. It's about five of them at the moment who probably know these values really well because they've been around the longest and they know what this looks like and what it doesn't and I think instinctively like a kind of

tribal knowledge there's something that they pass down and that they are using when they review code and they know what is open VAS and what isn't open VAS and it does go back to some of those principles of being able to run anywhere, any code, any scale, having things like a function store, a templating system to make that simpler to remove duplication, but also as far as possible to go and say, right, we're doing this by convention, not by configuration first. Let's go with connection to the next part of this community. And I think that the one oddity I see on your page, on your homepage is you've got Twitter, you got Slack, but in the middle of those two is LinkedIn, which is not normal for groups to sort of organize.

Why did you choose LinkedIn? And what do you do in LinkedIn over, say, Slack or Twitter? How do they differ? Yeah, so I can't remember how many followers there are on the Twitter account at the moment. I think I still have more of my personal. Slack has around 1300 people and LinkedIn has almost 300 people there. And that's something that I'm really proud of because it's only been going for about four or five weeks. And so one of the things that I do and I'm always thinking about is how do we keep the traction growing what channels do we need to be using right how do we keep people from falling out the funnel or at least from converting some of the dozens of people coming into the

community or seeing the project how do we get them to the point where they're really engaged for the project they may be giving back or they're becoming a promoter and they're helping the project to succeed and be sustainable. And I was reading a book called Traction. I think I have it up here somewhere. And one of the things he says is don't keep hammering the same channels that you've been hammering and getting successes off because you may be able to get the same successes in other channels. And some of, I think it's quite quite an old book some of the stuff in there we don't

really use anymore like stumble upon or some extent dig stumble upon huh stumble upon i haven't heard that in a while a long time i love asa love stumble upon yeah right but that's where i got the idea of linkedin but you know it's like getting a great encyclopedia from the library that's 20 years out of date. There's still a lot of principles in there that you can learn from. So I thought I'm going to try LinkedIn. I have a pretty good community there of people that are following my posts. Something that I've learned probably over the last three years is how to use social media to promote and amplify content, how to work with partners and other brands to cross market. And so the LinkedIn group at the moment is a very concentrated way of just keeping people up to date

with what's happening, telling them what the developments are. And not all of the people that are into the project are necessarily developers. You know, it's a much broader appeal than it did in 2016. So Twitter is probably the broadest kind of reach at the moment. But I think LinkedIn is just a little bit more concentrated. People are there because they want to be.

It's a group that you have to join and has to join. And then Slack, you know, I think we all know that Slack can be a double-edged sword. It can be overwhelming. It can actually be a really great experience at times as well. So I wanted to try and make sure that people knew what we were doing in the project, that they had a way to connect. And so far it's been really positive and I'm hoping it's just going to continue to grow. And what I find interesting about this is how you mentioned how you're personally using posts, which I'm curious about just because so many people I hear from that are developers, engineers, or in software, aside from, say, on the business side of things, you know, the relationship side of things, which everyone's in relationships, but developers are so inundated with requests for new positions and just recruiters yeah recruiters it's just terrible you know i mean it's great it's it's a good thing because it assumes there's lots of opportunity which there is but it also could be kind of cumbersome and the fact that you're just constantly being bombarded

you know you're you're you're overtaken so linkedin generally isn't the best place you want to like camp out at so it's sort of countercultural in a way for you to be in your position the founder of this project and leverage linkedin in a unique way with you mentioned the book uh traction is that what made you think hey i should try out something different and that that thing different was linkedin yeah and i'm curious also as a follow-up to that is how you personally use posts on LinkedIn. Yeah, so as the lead of the project, there are certain things that I feel I need to do to make it successful.

And there's different elements that in a regular business would be played out by different people or different departments. You would have a VP of marketing whose job it is to find these channels and and put this content out there and believe you me they will be using linkedin they'll be using twitter they'll be using facebook and they'll have scheduled posts that go out at different hours of the day maybe you guys have that as well and so no i don't i don't think it's odd at all for me

putting that hat on to go out and actually pursue those channels and um and broaden the reach of the project and at the end of the day there's 300 people there now they seem to be a group of people that are really interested in the project or interested in the topic and it's a it's an opt-in group and people can can come and go as they will again it's a bit of an experiment open files itself was an experiment. I didn't think it would get to where it is now when I first started hacking around with it. What I like about the LinkedIn group is the focus of it.

You mentioned the numbers are smaller at this point, but in this current state, it seems what it is, as you mentioned, Twitter's more broad. Slack is obviously real-time. It's got some pros and cons to it but to truly focus on say a a core group 300 people even if it goes to a thousand in the next couple months it's still you know fairly focused for you to be able to communicate with them is this a maybe going a little deep here on this but is this a broadcast system i've never really used linked in this way. Is it where?

So I've, I created a group and the group basically says that it, you know, it's about open FAS, join to share your knowledge, find speakers, get connected with the community. The rules are collaborative, friendly environment, share knowledge, build community, encouraging others on their open source serverless journey and that's really basically how it's been used if you're not in the group you don't see the posts if you are there's a feed and there's a guy um scott rosenberg out of israel and he posted um about a webinar he was doing with vmware there's another post from kieran Smith at SolarWinds who had just written a template for Swift. And then

there's something from where I did a partnership with Spotinst and the blog post is posted there by their guy as well. Yeah, so it's definitely an invitation to the community to say, hey, share what you're doing in this world. What's up, what you're up with with giving a talk

or presentation or whatever. It's kind of like that, but then also threaded conversations beneath it too. Yeah. There is a, a community.md, a markdown file in the FAS repo, and that is absolutely full of hundreds, if not thousands now of blog posts and events that are coming up and then some industry awards and notable kind of mentions. And if anything, it's just one way of bubbling that up. For instance, in France today, at DevOx France, Philip from GitLab and a friend of mine, Laurent, were talking about OpenFaz. I think they spent something like three, I don't know if this was the one where they spent three hours going through just about everything there is to know

about it. And it seemed like they had a great talk, but it's very hard for people to know what's kind of coming up. And so this is something that I do is kind of seek out who's going to speak or who's speaking and help them amplify it and get more people to turn up if I can. Let's talk about community financial support because as you mentioned offhand you are doing this indie full-time now previously you were doing it at VMware but you're out on your own trying to make a go at it what's your ideas around sustainability I know you do have a Patreon also you have an open collective so there's a few things out there where people can support the project financially but tell us bigger picture is this all experimental do you have a path towards monetization of some

sort of cloud offering what are your thoughts on that yeah that's uh that's a hard one and i don't fully know the answer to that as i said before there are half a dozen buddies of mine we're all in the same boat where we're these open source maintainers. Projects we're behind maybe have a lot of traction, but there's not a clear way unless you're perhaps a behemoth kind of enterprise company to actually go and support a project like this. So some of the models that I think we've seen in the past are providing enterprise support where it can help a company sleep better at night to know that there's someone

they can call. Another model that isn't actually necessarily helping the project but may bring funds is what Spring Source did when they started out which is training and consultancy. I think we're starting to see a move away from OpenCore with Chef, for instance, open sourcing all of their components for their enterprise add-ons. And then really, it's like, well, okay, as a SaaS, do you want to compete on price? Because there's no way you can. And so I'm not probably the best person to ask about how to monetize open source software and how to make it sustainable but one thing that i do know

is that whatever we're doing here and all the things that i've been trying have helped build a really healthy thriving community that really care about the project and have created something that's now in there basically on the critical path for for some very large companies and there's some um even bigger ones that are doing pocs with it at the moment that i can't give names for well let me ask this so close your eyes for a second and imagine three years maybe five years that's probably too many imagine three years down the road and OpenFaz has been extremely, extremely successful. So all of your goals being met. What does it look like three years from now? What does success look like if you had it your way?

Again, I think that's a great question. It's also a very big one. For me, where I am now is that I want to help people understand serverless and Kubernetes. I want them to help them solve problems that they're encountering on a daily basis. It's something that I'm seeing more and more. And so the compass direction for me is really the values of the project that have been defined. There's some chunky stuff on the roadmap that I'd really love to get into, given enough time and resource.

But really, I think there's a potential future where, as I said earlier, this serverless 2.0 idea comes to fruition on a larger scale where it's that container image and that basic contract that is what is portable between every cloud. And it may not be that it's running only on OpenFast. There's a provider that has been put together recently by one of my contributors, Ed Wild, and this can target Fargate. There's another one that we kind of came up with the idea for over lunch at Christmas, which was FAS Lambda.

And you can literally build the same function and deploy it through exactly the same tools, monitor it in the same way. And that same function can deploy to Lambda and Kubernetes exactly at the same time. So I'd love to see OpenFAS, the packaging layer, and the runtime, the interfaces, become something that's very broadly adopted and is helping companies big and small to start solving these problems and make

complex tools like Kubernetes more accessible. Maybe the other question should be whether or not this should be an open source project that makes money because whether it can is one thing, whether it should as another. And it seems like a lot of your success has been focused on what makes the project itself thrive. Not so much how to get people to pay to use it or sustain it or maintain it or whatever word you want to wrap around financially giving to it. Yeah.

Okay. Cause you've said yourself, you don't feel like you're the person to to do that in particular i'm wondering you know does that mean bringing somebody else on the email doesn't have the answers maybe so no i don't know i don't know if that's what i said um at all i think what i said is that it's very challenging to make money commercially or get a commercial proposition of something that's given away completely for free and super easy to use. I don't think the project itself can make money.

A company could potentially have a go-to-market strategy building a product around the project. But this is something that while I was at VMware, I really defined a bit further and I think they reinforced as well is that projects are not products. Products are products. Now you may create a distribution of OpenFaz that is a product that is a paid support, has a paid support contract with it. Companies will be willing to pay a corporation, either one person or three or five or 500 to support. But on its own, it is not a product. It's completely free. It's MIT licensed. That means that it cannot make money. It can make money for you when you use it in your business. I was speaking to a developer at Ytel. I don't know if you're familiar with the company out of Orange County and they sell sim cards in

stores one of the things that was kind of a choking point for them was that they're getting you know less less than excellent customer service because they'd click the button the system would kind of choke and that was because they were trying to run these things synchronously well by moving into OpenFaz and using the asynchronous capability that's built in trying to run these things synchronously. Well, by moving into OpenFaz and using the asynchronous capability that's built in, they could run these processes in the background and free up the system to do the next thing for the customer. And so for them,

they'd totally be making money out of that, right, with their customers. But no, I think we've got to be really careful when we use this term project and product and monetization. The project does, from my perspective, would really benefit from sponsorship, especially from some of the larger companies that are making use of it. And it's profitable for them and they're making big savings. But, you know, this isn't always the way the world works, as you'll see with some of the other big open source projects.

I did start a Patreon campaign and try as I may, I've still not got anywhere near even minimum wage levels of someone flipping burgers, not even 10% of that, to probably to be able to support even myself like on a minimum wage right and that's how hard it is so i i don't have all the answers but the project is doing really well there are people that are really committed to it and it has growing you know growing traction we spoke about linkedin i get several people from directors to vps reaching out every other week saying how they're evaluating open fares maybe using in a poc or they want to

kind of connect back to the community in some way so if there are people out there who have ideas concrete ideas i'd love to know but this is something that i'm working on and i'm trying to figure out at the moment so we've asked you to look down the road. We've asked you about sustainability and money matters and all those things. Let's zoom in a little bit and talk about tomorrow. As the maintainer of this project and the leader of the project, when you wake up in the morning, how do you decide what to work on? What's most important and what you should be doing today or tomorrow? So when I wake up, I usually ask my wife if she slept well. And then the next thing that I do is I'll turn over to my phone and I'll check my emails to see if anybody had any

problems with OpenFast. Then I'll check the Slack to see if there's anything important. Then I'll check LinkedIn, then Twitter, and then I may get up and go on and start the day at which point maybe drill into some of that stuff. And so at the moment it's very much like the, what, you know, my time at VMware was very similar, it's putting on the maintainer hat, putting out any fires that are, that are there keeping the lights on. And, um, part of that is reaching out to this network of contributors through various Slack channels and other mediums,

seeing what PRs need merging, which ones I'd promise to review, who I need to spend a bit more time with, if there's anybody that I need to schedule a one-to-one call with and give them some feedback or ask for some feedback for myself. And so I try to kind of run this as for all intents and purposes as a business that hasn't quite figured out a monetization strategy yet. And that can be hard. And one of the ways that I try to manage this is to try and stay out of the minutiae where I can and to to delegate delegating is very hard when the people you're delegating to are volunteers and sometimes I feel bad for asking them to do things or to help out

on stuff because I know they have their own full-time gigs and lives and and so I'm quite sensitive to try and not ask people to do more than two things at once and to try and ask always ask them how their day was or how they are as a person if I know there's something going on with them I'll try and kind of foster that relationship I've also sent out swag and handwritten notes to people and this isn't like the ones that I've seen on Twitter from a well-known corporation recently where they're all just cookie cut and it says at the end please share on instagram this is like really trying to foster that relationship with that person i think it's interesting because you said you tried to avoid the minutiae but you started

off by describing most of your day is minutiae is dealing with minutiae and it makes me think that maintainership and leadership are really two separate roles. And oftentimes they're done by the same person because of the state of the world, right? And that's the same thing with small business or startups. You end up wearing all of the hats. But you need to be able to dedicate time to not just maintaining the status quo, but actually pushing OpenFaz towards that future that you see for it. And I'm sure it's incredibly challenging to do that when you're putting out the fires and

maintaining the relationships and answering the questions and delegating. Yeah. So, you know, when I say minutiae, what I may mean is that if i've delegated to somebody to add a certain cli command that i will leave leave them to go and do that and they may be working with someone else and if they've reviewed it and they're happy with it that it's not going to be an in-depth review that comes from me next it's going to be an eyeballing and a nod to it and maybe you know have you thought of this and then getting it in there an example of that would be recently there was a blog post on apache kafka that we did and i've been wanting it for a long time and this is one of the challenges with delegation and delegation to volunteers is my sense from the leadership perspective was that this needed to be timely

and probably was overdue however delegating that to somebody then meant that it's on their schedule. And one of the things that helped me a lot was there's a kind of tic-tac-toe sort of spectrum graph where it shows you important, urgent, can wait, can be done by me, can't be done by me. And that's the kind of thing where sometimes as an open source maintainer and my co-contributors get into this trap of thinking that we must always try to get somebody in a community to do something first. We must always find somebody to do a quick PR when in fact, this is one that falls into the quadrant of

is important or is urgent and can be done by someone else but actually it's time to just commit and get that done and some of the best time that I have is where I can sit down and get into a problem and work on it and produce a feature and it can be a very frustrating experience trying to coax this out of other developers over a process of weeks or months when i know dedicating an afternoon i can have it completely deployed and tested right it seems to me like your schedule is proactively planned but reactively executed meaning that you're willing to react to the needs but you're proactive in how you approach it. I heard of this blog post from someone at Y, I think Y Combinator, the maker and the manager schedule.

And I told my friend and he said, don't tweet about that because people will think that you think they're stupid because it's been around so long. And somehow I never happened upon it. And he said, you are basically on, you figured it out. So I don't know what you're talking about. When I read the post, I'd realized that I'd adopted the best of both worlds, where I was running the maker schedule and cranking out the features in the code. An example would be like a OAuth in OpenFast Cloud that needed a lot of concentration.

It took about two weeks to develop. But also running the maker schedule especially now you know I've been interviewing with about five six different companies that are very interested in in doing something together now I I know just as it says in the post if you schedule a meeting at 10 a.m or 3 p.m that's gonna mess up your whole day for whatever maker stuff you had planned just subconsciously it seems to do that so try and pack those meetings into a single day. It will be exhausting, but it's more efficient. And then I know that I've got that other time free to do these contributions and actually make sense of the world.

It's a great constraint. It's interesting that you pack it all in one day because, you know, sprinkling across many days seems to be the most efficient just because it gives you the most variance and flexibility. However, in practice, when actually executing on your plans, I agree that if you've got something on your calendar, you're mapping some mental space around that and or just the rest of your day is somewhat mapped around a couple different key moments in your day. And all bets are off, so to speak it's just interesting to to look at this jared i like the point you made about maintainership versus leadership and i think that uh when you look at a project like alex says here and the success of the community it's really important and really wise to drill into those particulars because it's not often well i guess i should say it's not often but i guess it is not often but it's not often that you get a chance to sort of like hear this wisdom and you

know i think it's part of our jobs as this show to do what we can to get alexa share that with the rest of the world because we want more open fads it's just not you know competitors we want we want other things like it you know in ways that's being run. Yeah. Get your own name. That's right. Get your own name. You know, I would love to see that too. And I think part of the community I've built here was almost bootstrapped from the community I'd built as a Docker captain, building tutorials and blog posts and, excuse me, working with the ARM devices and Raspberry Pi. And they still have a place in the OpenFaz Slack. And one of the things that I've been starting to do is try and think, can I spin out a new community

that just has a focus on working on some of these interesting open source projects? But again, this goes back to that strange position between having the maintainership, or let's say the execution side of things and the leadership. The leadership is a thing that has to think about, should we have a strategy to pursue new channels and engage people through LinkedIn? The leadership side of thing has to think about who should be part of the GitHub members next, who, you know, what opportunities need to be pursued, what meetings need to be had, that kind of manager schedule. Then you've got the execution side of things

that's very, very much necessary too. What I'd love is to have the ability to sort of grow out those functions and have other people take it on full time. That's something that as we explored earlier, needs to have a very clear business reason for that happening. For now, I'm doing what I can. And a lot of people are working really, really hard and really care about this project. And is it going to be the one that's winning in three years? I don't know. But I do know that there's a great group of people behind it

that are working really hard. And there's other stuff that we do too. And I think that's really the key thing is that I put a project together over the Christmas holiday called Inlets. It's a HTTP tunnel. And the reason I built it is because I used to work at an enterprise and we couldn't use ngrok and things like that. And my team at the time was also working in an enterprise and they couldn't use ngrok. And I wanted to create something that would solve that problem for them of being able to test webhooks and that kind of thing. All of the lessons that I'd learned over those past three years of development and community management, and one thing, another, I was able to replay

in two weeks. All of those learnings that had taken me so long to grasp, I was able to replay them very quickly and also engage my existing community in that project. And then posted it into Hacker News and got two and a half thousand stars in a couple of days. Kudos to you too, Jared. You logged this in February and let's expose your local endpoints to the internet. That's right. Good job. Well, thank you. I think that was the first time I saw Alex's name and then I saw it again on OpenFast and I was like, all right.

Yeah. Two times a charm here. He kind of in happenstance told you what he desired for the next few years of the future. That question you answered or that you asked that he didn't quite answer, which is kind of cool because it sort of came out naturally. There you go. Like your desires are to grow out of those roles, but you're trying to figure out how to best do that, which is interesting.

Well, I thank you so much for your time today. It's been an honor to talk to you. It's definitely interesting to see where this project is going. Any, any sort of shout out you want to give here at the end, any core contributors that are just like killing it, that are helping you on a daily that you want to give a shout out to before we

tail off. You know, there's so, there's so many people that it would almost be unfair, but I want to say a big thank you to Richard Gee that's helped me out right from the beginning, to John McKay over in Ireland, Ed Wilde, Burton Rutan, Lucas Rosler,

and all of the other people in the community that have been there for me and have been there in the community that have, you know, that have been there for me and have been there for the project and have helped to make it what it is. Very cool. Thanks again, Alex. It's been awesome.

Thanks for having me. It's been a pleasure. All right. Thank you for tuning into this episode of the change log. Hey, guess what? We have discussions on every single episode now. So head to changelog.com and discuss this episode.

And if you want to help us grow this show, reach more listeners and influence more developers, do us a favor and give us a rating or review in iTunes or Apple Podcasts. If you use Overcast, give us a star.

If you tweet, tweet a link. If you make lists of your favorite podcasts, include us in it. And of course, thank you to our sponsors linode go cd and rollbar also thanks to fastly our bandwidth partner rollbar our monitoring service and linode our cloud server of choice this episode is hosted by myself adam stachowiak and jared

santo and our music is done by breakmaster Cylinder. If you want to hear more episodes like this, subscribe to our master feed at changelog.com slash master or go into your podcast app and search for Changelog Master. You'll find it. Thank you for tuning in this week. We'll see you again soon. because you've listened all the way to the end of the show got a little preview here for you of our upcoming podcast called brain science this podcast is for the curious and explores the inner workings of the human brain to understand behavior change how about formation

mental health and the human condition. This show is hosted by myself, Adam Stachowiak, and my good friend, Muriel Reese, a doctor in clinical psychology. It's Brain Science Applied, not just how does the brain work, but how do we apply what we know about the brain to better our lives? Here we go. That applied brain science really stood out to me because I want, I don't want it to just be data. I want you to go, how can this fit? What can I take away now? How am I going to change?

And that that sort of is where you come in more. And even some of the questions like, so like, I want to ask you, what are some of the most challenging things working in the tech world when it comes to relationships? Probably the most important one is isolation. More and more of the world and companies are being, for good reasons, they're being okay with what they call distributed teams. Yeah. And that means that you and I, we work for the same company, but you work from your home office. I work from my home office. I might go into the office a couple of times a week if I live local, but even if I live in San Francisco, I'm still probably a remote worker, even though I can hop in an Uber or hop on the train or whatever and go into the office and be there in a half hour. But why waste the time?

You know, and this is where I would revisit what I want to talk about with resonance. And that whenever we're learning, no matter what thing, it's really helpful when we get feedback that's both immediate and specific. And so when you're by yourself and you don't have any interaction with other people, how can you get any feedback? I mean, you're losing most of the nonverbal communication and you also don't have all of the voice inflections or facial expression. Have you ever tried to, you know, be sad, feel sad and smile at the same time? Try it.

It's pretty hard. Right. Because facial expression is exactly what's involved when it comes to empathy, which is relationships. I was reading a research article recently and it talked about, you know, how couples who are together a really long time end up sort of looking like each other. Never heard that. Yeah. And so what they've looked at is when we actually empathize with other people, facial expression is really key within that. And so when you empathize with the partner you're with over and over and over again, your face begins to make the same creases and facial expression as it relates to where somebody else is emotionally.

Wow. Right? Say it is. So that's, that's creepy. Well, they've, again, this is sort of the hotbed when it comes to neuroscience these days is mirror neurons. And these mirror neurons are what are involved with empathy. And so mirroring, meaning I get another person's emotional world. And so one of the research studies looked at Botox. And what they found is that Botox, because it actually assists in paralyzing facial muscles, then you can't contort your face, so you don't get wrinkles,

but actually levels of empathy go down. Right. Because your physical appearance can't reflect your inner appearance. Yeah, you got it. And so when you're working in these remote locations, it might facilitate better work or more focus. And it allows people to be distributed and to capitalize on the talents across the country, right? Yeah. Wow.

So that's like a treasure trove, in my opinion. Talking about in a scientific way, you know, not just like, hey, this is my opinion about all the cons of that. Because I think what we can do is still have remote work, but do it in more healthy ways. Because I'm fully, I mean, I've been self-employed remote worker since 2006. Now I'm a unique animal. I know that. My wife knows that. Right.

And I'm a unique animal. I know that. My wife knows that. Right. I'm fine with it. I'm a good human being, but I've got some flaws and I'm willing to accept and share those to some degree. And I think the problem is we just lack maybe a more purposeful or intentional feedback loop. Yeah. Which I think is super important to being able to operate in this world in just good ways. I don't know, healthy ways is probably the best way to use in this show context is healthy ways. One of the things that's fundamental, I would say, to being human is change, right? And so sometimes people come in and are really key in our life for a period of time.

And then things change. Either we grow or they grow or they change in a different direction and then the relationship changes or that feedback loop gets modified in some way that isn't always a bad thing it's just going my sense of choice actually is a critical component when it comes to feeling good about my life. If I feel like everything is sort of outside of me and I don't have any charge over it, like I didn't choose to work in a more remote location or I didn't choose to go to school or I didn't choose this person, then it feels far more oppressive as opposed to I actually participated in the outcome that I'm actually

experiencing. So I then also have more charge over whether or not I want to change it. I think this feedback loop process that we're talking about here is super common to developers, you know, from people who write code to people who plan and to engineer and to manage and lead. There's no one in the software process that doesn't understand the feedback loop. And the reason why is because in product development, they have this concept of agile. And basically, it means you produce something, you put it out there, and you expect the feedback loop to happen in order to gain insights and course correction to then release another

version of it that continually and iteratively becomes more and more improved. So this whole process in day-to-day work in software is normal. And I think it's interesting how it can apply to their lives and people's lives, you know, to take the same importance of a feedback loop, for example, and apply it. Right. Well, so this is very much how it goes in relationship, which is why there is an importance when it comes to sort of things resonating. You ever walk into a room or an interaction with a couple other people and like something just feels wonky or off?

You're like, I can't put my finger on it. Definitely been there. Right. You're like, I can't put my finger on it. Definitely been there. Right? Well, and so to be able to identify that in relationships and even go, wow, I need to, I'm experiencing this person in my world with the limited interactions that I have with them. It hasn't really resonated with me. And so I don't get good feedback. So now I'm going to be more defensive because I feel as though there's a threat. It doesn't necessarily mean the person is threatening. However, my brain is going to tell me, hey, we need to be more protective. We need to do some strategies so that you're not fully exposed. like this uh i would say as of late is because if you ever watched a tv show or a movie where

the you know the narration the storytelling part of it they expose a character in a certain light and you may dislike that they may be a villain or villainess right sure but the moment they turn the story to their backstory and why they are the way they are or why they're acting the way they're acting. Yes. You then kind of fall in love with them and you're almost rooting for them. Right. I feel like that's the same thing that happens day to day to our lives is that, you know, there are people who seem villainous or not for us, but we don't understand their backstory and why they are the way they are for us to have and employ that empathy that's required to have this dance, as you say, this iteration of relationship. You know, we just assume they are who they are and we project, you know, our worst fears onto them and they become true.

Yes, you got it. This is why in the absence of, you know, a face, I don't really get to engage with people in the same sort of humanness that we are all in. And so you're exactly right. I mean, over and over and over again, because you can identify and go, oh, that's why they're harsh. Or, you know, I recently had an interaction I had shared with someone that I was a competitive gymnastics coach for a number of years. And so somebody thought that my response to them when they were really struggling was kind of harsh, but they remembered that I had told them I was a coach for so long. And they're like, oh, this is just another side of her coming out.

Right. And I'm not sure I prefer it, but I get it. And then it switched for their reaction because then they're like, oh, wait, we're on the same team. She's not trying to like oppress me or fight back against me. She actually is helping me trying to get me to where I want to go. My wife and I, we've learned this, this concept of goodwill, right? Yeah.

I can take your feedback or your criticisms in a different light if I know that you have goodwill for me. Yep. Meaning that you're not trying to harm me, that you are for me, not against me. And sometimes change, as we all know, is painful and can be painful.

So sometimes the necessary feedback and or criticism that can influence that change can also be painful. But I can accept it differently if I know that she or they or whomever is in the scenario with me has goodwill for me. Whereas if you know that they're not for you, then you obviously take it a whole different way. And that's an okay thing. But we often are in relationship with people that are giving us crucial feedback and we need to have that kind of that lens. Like it was significant in our marriage to understand, hey, I know there are times when you give me feedback, I am not happy about it, but I know you have goodwill for me. So therefore I calm down. I listen. I,

you know, I take that in and I process it, whatever, but I take it in a different way because I know that she's for me and not against me. Yep. One of the key things when it comes to change is a sense of openness and even relationally like of going,

I need to be able to see how somebody else responds or how they're feeling as based on their perspective of what they're going through and not just my perspective of their perspective. And so this goodwill is like, I believe that we're on the same side and that you're not trying to make it harder for me. But so I can understand if I were sitting where you were sitting, had the background that you had, why you would have taken it in that way. And then I can provide an opportunity to clarify or create

more connection, even when it doesn't feel good. And I honestly think this is so much of what's missing in people's relationships. If I look at relational interactions through the notion of conditioning, wherein I get a sort of hit of dopamine, feel good feelings because I went to a person, I had a conversation that didn't necessarily feel good, but there was openness on both parties to hear one another's perspective that it actually then reinforces like, oh, when I go and I have this exchange with people, I feel better. So now I'm going to go and engage with other people and get the feedback, even if I might not like the feedback, because now I'm buffered and I'm not alone in this. And I somebody else sees my world.

That's a preview of brain science. If you love where we're going with this, send us an email to get on the list to be notified the very moment this show gets released. Email us at editors at changelog.com in the subject line, put in in all caps, BRAIN SCIENCE with a couple bangs if you're really excited. You can also subscribe to our master feed to get all of our shows in one single feed. Head to changelog.com slash master or search in your podcast app for ChangeLog Master. You'll find it.

Subscribe, get all of our shows and even those that only hit the master feed. Again, changelog.com slash master. Thank you. you