The Data Stack Show - Data Debrief: The Highs and Lows of Open Source Projects

Episode Date: November 5, 2021

Eric and Kostas break down further topics from episode 60 about stream processing and open source projects.

Welcome to the Data Sack Show Debrief. Costas, one thing that I really, really appreciated about the conversation, well, two things. The first one is pretty straightforward, which is I really enjoyed hearing Ashley talk about making product decisions because you thought it would be fun. And in the world that you and I live in, it's sort of a product-led growth mentality and sort of rigorous requirements around analytics and all of these other things. It's just really fun and refreshing to hear someone say they added a feature because they

thought it would be fun. And the next one is, I think the whole conversation around having opinions, strict opinions is really interesting. And it's certainly a very difficult topic when it comes to event ordering in a stream processing system, which is, I mean, something that of course we deal with in a very critical way with RudderStack. But I loved those two components of the conversation were great. Yeah. Yeah. I was thinking about that part of the conversation.

And I think that actually, like, that's the very, very important point around engineering in general and open source in particular. When you want to lead like a complex engineering project, your main responsibility, I believe, is like being the person who is going to be very opinionated about the trade doors that have to be made. Because at the end, what differentiates one project from the other or what is the essence of engineering is like managing trade doors at the end, what differentiates one project from the other or what is the essence of engineering is like managing trade-offs at the end. Like why I would sacrifice performance for transactions,

for example. Because you cannot have both at the end. There's no way that you can have both. So there's a trade-off there. Who's going to decide about this trade-off? Someone has to do that, right? So being an open source project leader or a technical leader in general, it's a lot about having opinions about the trade-offs that have to be made. And I cannot stop thinking, I think there is the archetype of this, which is Linus, the guy who is like running like the everything around Linux. So he's the most, the most opinionated person that even had like at some point to do therapy for that. Yeah, yeah, yeah.

Oh yeah. Well, I was trying, as you're talking about that, I was trying to form this really rich analogy with like relationships and trade-offs and then i went further down that path and it's like well your responsibility is to take ownership of the opinions about the trade-offs and i was like yeah it doesn't really translate to relationships super well yeah yeah so yeah that fell apart in my head yeah that was like a very interesting point in our conversation outside of this like i mean it's it's amazing when you realize how much a person can create when they are passionate about what they are doing.

Because if you think about it, we are talking about like a system right now. Think about the value in this world that Ashley as a person has brought because he's passionate and he's having fun when he's writing software. I think considering like, okay, we are not solving like, I don't know, the global hacker problem or whatever, but still like if you see like how much time is saved, how many companies are using the software, right? To run like very complicated infrastructure that then affects like the experiences

and like the life of like other people. And all that like stems from one person. So I think that's another part of like the beauty of like talking with these people that are making like open source projects. Yeah, it's I don't know, I find it might sound a little bit romantic, but I find it like, very, very impressive. Well, I think a bent towards the romantic is appropriate, is an appropriate perspective. Especially with this logo, right? Yeah.

Well, if you scroll to the footer, it's like a blobfish heart for like the make a donation. And I mean, it is really ugly in like the best way possible. I will say, last thing, cause we need to land this plane. Reading through the docs and the site, I just had this sense like, this guy's really funny. And it was really great to meet him

and really see that his personality is shining through the website and the docs and the videos and everything. And so that was truly enjoyable. And he's just a great, he has a great dry sense of humor in general. So that was great. All right. I think we kept it under five minutes, maybe not,

but that's debrief number three and we'll catch you on the next one. you