Coding Blocks - Designing Data-Intensive Applications – Multi-Object Transactions

Episode Date: February 6, 2023

It's time we learn about multi-object transactions as we continue our journey into Designing Data-Intensive Applications, while Allen didn't specifically have that thought, Joe took a marketing class, and Michael promised he wouldn't cry.

You're listening to Coding Blocks, episode 203. Bop, bop, bop, bop. Is that a big one? Oh, wait. No, I forgot. I'm sorry. Hold on. Let me start over.

You're listening to Coding Blocks, episode 203. That's fair. Yeah. Integrity. Yeah. Yes. I think, is that too NPR-ish or is that more late night DJ? Well, Outlaw just rolled out of bed.

That's why we got the bad boys going on. Subscribe to us on iTunes, Spotify, Stitcher, wherever you like to find your podcasts. Visit us at CodingBlocks.net, where you can find show notes, examples, discussion, and more. Send your feedback, questions, and rants to comments at codingblocks.net and head over to the Twitters at Coding Blocks. Or head to www.commentary.com for social links at the top of the page.

I immediately regret my decision. I'm Howard Rogan. I don't know who that is, but it was funny. I'm Michael Outlaw. And I'm Alan Underwood. You don't listen to the Howard Rogan show?

I've never heard of Howard Rogan. I think you've mentioned him before. Was he the ASMR stuff? No, I just said... That was something totally different. No, the portmanteau of Howard Stern and Joe Rogan. I was trying to do my impression. I guess I didn't do very good if you didn't recognize those voices.

Yeah. Oh, man. All right. So, hey, we are actually going to talk about some programming stuff today. Maybe in normal voices, maybe not. So we're back with Designing Data Intensive Applications. And today we're going to be talking about single and multi-object operations related to transactions.

So before that, though, we got some reviews that Outlaw needs to read off for the world to hear. Yep. So, you know, Dombbell30, now you regret mentioning your myths of the late night DJ voices, don't you? You heard that intro and you're like,

ah, I shouldn't have said it. Oh, is this where this came from? We have somebody to thank? That's amazing. And then, Tun Tun Tun Too, thank you very much. And, oh, I know, I messed up. That one was a repeat from last week. So, yeah, those two are new reviews.

All right. Excellent. Thank you much. Don't forget. Leave a comment on this episode. You might want a book. And speaking of book, I just noticed this book was actually on Audible. So I picked that up and I've been listening to it.

It's pretty great. I think you said there a little bit quicker about going through the books. Yeah, yeah. You can actually listen to the whole book in about as long as it takes to listen to an episode. I think you said they're a little bit quicker about going through the books. Yeah. Yeah. You can actually listen to the whole book and about as long as it takes to listen to an episode. That's, that's,

that hurts, but yeah, it's much shorter. In fairness, the screenshot you sent were like, just like the very super small paragraph sections. Yeah. Not like the chapter.

They were. And in fairness, also listening to the technical stuff like that you got to be hyper focused because at some point you'll just find yourself drilling on your keyboard and having to rewind half of what you missed anyways right yeah listen to cutting blocks while you're eating your taco bell or whatever that's right we keep you awake del taco your. Your correction. Del Taco. Hold on. Hold on. Taco Bell versus Del Taco. Taco Bell, man.

I think they win. Yeah. Del Taco. They brought the Mexican pizza back, man. No, they brought the Mexican pizza back. Del Taco is way spicier. Seasoning is better. Del Taco is greater. Del Taco greater than Taco Bell.

I mean, if we're talking about crappy fast food tacos, then... Yeah, right than Taco Bell. I mean, if we're talking about like crappy fast food tacos, then. Whoa. Yeah, right. Taco Bell versus Del Taco. Jay-Z, which one? I mean, it's obviously Taco Bell's the winner there. It's got that microwave flavor that's hard to replicate. The corn to cheese have that cardboard crispiness that you just can't replicate.

The beefy five-layer. Cardboard. No cardboard no dude the beefy five layer burrito if you haven't gotten it you're missing out you need to go get that one yeah that's good at any rate joe was beating himself up because you know the audio version of the book uh you know every section of it was like two minutes but those weren't the chapters so like like at first glance, he was like, Oh my gosh, they're fast. Yeah. Yeah.

There's still probably a little bit quicker than we are, but they don't talk about Taco Bell and Del Taco. It's missing out. So, Hey, one other thing too, uh, we've sort of started remembering to mention this is,

Hey, if you want some stickers or anything, we have a swag page, go to coding blocks.net slash swag and you can get a get an address where you can send an envelope and we will send you stuff back now you said or anything so if they were like i want a pony we might have some some dryer lint left over that we could we could share something i don't know we have all kinds of things right so there could be a surprise

come back wow you know yeah yeah uh also before i forget i want to mention that orlando code camp is coming up it is a free event and get lunch too in uh sanford florida you know basically central florida area that you should go to it's going to have a bunch of different talks uh like the talks are going to be announced soon uh it's a really awesome event went on hiatus for a couple years doing like kind of peak covid but it's back now better than ever and you should go check it out if you're anywhere nearby or you feel like making a road trip hey and and i think all three of us can attest that is a fantastic event they put on there um one of the better code camps that i've been to and we've been to several different places so uh if you're anywhere near it i highly recommend checking it

out it's fire i think you could say it's fire i think they said i hope they said otherwise i don't know that passed on last year who knows hello fellow kids i am now with the 411 i love music band with the 411 that's amazing yeah get the show started yeah let's do it we're talking transactions is chapter seven uh we're picking up with single object and multi-object operations and so uh you know single object we talked a little bit about last time uh basically being able to you know either do it or not do it and not have changes in progress that kind of get mixed up and in bad shape multi-object transactions need to know which readings reads and writes are part of the same

transaction and uh as we talked about last time, it kind of got started in the 70s with first relational databases. In that case, in relational databases, the transactions are typically handled by tracking the connection. Yeah, that was interesting. Yeah, unique identifier that you're able to use to identify. And then also having statements wrapped in with like a begin transaction and commit transaction down to the end to kind of like put it in the logical block to mark the transaction. Yeah, so the thing where it's being handled by the connection, that makes a ton of sense. They said typically that's how it's done.

So I was kind of curious, like if there are other ways that it's maybe done in there, I guess they could do it in a stateless way. I'd never really thought about it, but I don't know. Just when they said it was typically done that way, it kind of jumped out at me like, huh? I mean,

it makes total sense. It's the one thing that's usually stable for, for all the processes, all the, all the commands that you've issued. Yeah. Except that part where I was like, well, that can't possibly be good enough, right, in regards to connection pooling.

So you have one connection, multiple things are coming in, and that database, the IP connection to it might be abstracted you know because it might be all coming from a single api uh you know server or at least like some some known set some known quantity of them not necessarily just one but i'm sure there's some sort of unique identifier that goes along with those connection pools that that makes it act like it's a single connection, right? That'd be my guess. But I'm sure there's all kinds of little tiny operational things that happen behind the scenes there. But the thing that they call out next is

a lot of non-relational databases don't have a way of grouping these statements together, right? So if you update order details and update um inventory they don't you know whereas the the previous with the transactions would do that with the connection this this kind of doesn't have a way and and so they a lot of times just don't even handle it they they did kind of go just rewinding one moment to my point though they did the other did address that in the footnote that where how did he refer to it about there being a transactional manager that would keep up with a unique transaction identifier. And then that way it's not bound. And if the client did lose a connection for any reason, it's not necessarily like completely lost.

But it is covered later in the book in more depth. Oh, that's good. So this is one thing where we've talked about this before. You have the actual hardcover copy of the – or not hardcover. You have the physical copy of this, right? Yes. I have the Kindle version of it, which I usually like way better. But one thing that is annoying when you're reading the Kindle versions is when they have links to like footnotes and stuff like it takes you away from where you are.

And it's not super easy to get back. There's like a back button. Yeah. So that's one of the things that's kind of irritating. And that's probably why I missed the footnote is because anytime that they have links to those things, I'm like, man, I don't really want to lose where I am.

Right. So you don't want to be taken out of the moment. Exactly. So, all right, well, just call me Michael, a footnote reader outlaw.

And, uh, I like it. Yeah. Just a quick reminder to you on kind of how, uh, a lot of, uh,

not new sequel type databases work. You remember if we talked about this uh early on like it uh one common way for uh for non-relational databases to work is uh to basically work on a quorum model uh relying on eventual consistency meaning your client is going to reach out to all the various primaries and blast out its update. And then once it receives enough of a signal back saying like maybe 7 tenths of the replicas have said, I got it.

And you say, okay, I'm going to trust the others to catch up via some other mechanism. I'm going to get out of here. And so you can imagine how that kind of thing doesn't really work too great with multi-object transactions where you need everything to be done everywhere to make sure it's in a consistent state. state just kind of doesn't really jive with that idea yeah it definitely makes things more difficult right because now your object or your order

details in your inventory are out of sync across you know multiple replicas so um that now they get back into the single object transactions and this part was interesting to me because it seems like it's obvious but then they call out some things so um single object transactions must be atomic and isolated right um so what happens if you're writing a block of data but the connection was interrupted in the middle of the right? That's one one situation where things could go wrong. What if there was a failure while updating an old block of data? So now you have a mixture of new and old. Like what happens then?

Reading values while in the process of updating. So, you know, outlaws in the middle of updating a record, I'm in the middle of reading a record. It's only halfway through. What do I get back as the reader, right? So those are things you have to think about. Yeah, there was also a comment in this area of the book, too, where someone thought that failure or that,

uh, the writing the block of data, but the connection has lost thing. Like it doesn't necessarily mean that writing the block of data, uh, like the, the, that could have been successful,

but then what do you do? Like, how do you return back? And there's later an example where, uh, they talked about like, imagine the author was using, um,

like we, we've been using our canonical reference of an e-commerce kind of site, you know, for a lot of the subjects that we talk about on the show, right? And in this portion of the book, the author was using like an email system as part of it. And he was like, he gives this example of, well, you wouldn't necessarily want to keep resending the email on a retry just because you weren't able to acknowledge a success the first time or something like that, right? So you can see where there is definitely complexity and even in this single object idea. Well, in that case, in that specific

example, he was talking about multi, but the idea being that something could have technically succeeded you just can't acknowledge the success right because the failure happened right after the success and so your connection to it to get to get the acknowledgement back failed so so what are you doing your application at that point right like and that's that's a tough one that's that's a harder one for, for the developers to work on, but hopefully behind the scenes, the database worked out actually completing the data. So you're not in a bad state. Right. And that's, that's kind of what they were talking about. I mean, again, you know, because of this book, right? Like when you're

reading this book, put your, you know, the hat that you should be wearing is not like I'm the e-commerce developer, but instead I'm the developer creating a database, right? Or, you know, like I'm going to create the database from scratch and this is how you might use it. And so as reading this portion of the book, I was kind of thinking like, how would you handle that section? Like, you know, you can't acknowledge the success. So do you assume that the app developer writing the e-commerce site is going to like re-query the status to say like, hey, is that product still available in stock?

Because I still want to try to place the order. But one thing where like, I was thinking about this, our canonical e-commerce reference kind of breaks down in this, at least in this area of the book. And maybe why the author chose not use it is in the e-commerce example. Um, like one of the things I was thinking of was like from an inventory point of view, like you can't, you're trying to update

your inventory and, uh, it fails. Right. But that doesn't mean that you don't want to sell it. You don't want to make it available for sale. So you still want the customers to be able to place their orders just because you're having some kind of technical problem. Because in the meantime, you still have sitting on your warehouse floor inventory that you want to get rid of, right? So you don't want something like this to stop it, right? So that's the problem. And, I mean, we kind of hinted at that in the, I think it was the previous episode where we were talking about, yeah, it was the previous one where we were talking about the transactions

where, you know, I kind of like, as we were describing it, it kind of dawned on me like, wait a minute, e-commerce would probably be a bad example in this case. Well, but that's where things get way more complicated, right? So what we're talking about here is in a database, right? Like a single database or whatever. The reality is, if you were worried about those types of things, you wouldn't just be doing a database, right? You'd probably have a queuing system out there that, that would get used for working against the inventory.

You'd probably have a queuing system for placing the orders as well. Right. So, I mean, maybe you hope that, I mean, you, let's say, let's say maybe, and it certainly sounds nice, but we've definitely been at smaller shops where that is not the case. And so, you know, uh, my, going back to my point of as an e-commerce type environment, you might be willing to – this orders flow through and then reconcile inventory later. You know, it's kind of like even in the physical stores, right, where they'll do, you know, the employees will take inventory of what's in the store after the fact, right? They're not trying to necessarily, I mean, they're keeping up with it in real time as best they can, like every time they make a sale.

But that doesn't necessarily make it so. And they're not going to not sell you something because they're like, well, hold on. My inventory says I have four of those. And I know that you have five presented in front of me, but I only show four of the computer. They're going to sell you have five presented in front of me, but I only show four of the computer. Right? They're going to sell you all five. Yeah, it's definitely one of those things that boils down to if you're an Amazon, you have things in place for all this, right? If you're a little mom and pop shop that's starting up selling your own widgets on Etsy or something, yeah, you're probably not going to have these things in

place, right? And then you're going to just face those problems when they come along. So, hey, so getting back to the reading, like all those weird problems that we talked about, like something fell in the middle of a riot or a middle of a reed, whatever, it's for that reason that all databases have to support single object atomicity and isolation. So I think that's probably even goes for distributed databases. Like you can't have something fail in the middle of a read and then replicate or in the middle of a write and replicate out to everywhere. So they basically say everything has to deal with that. Yeah, I actually went looking to see if there's any that don't.

I thought if all major databases have atomic operations, maybe there was one out there. Maybe someone had a use case in the wide world where they would want to accept partial values. Maybe just get everything they can. And even if it's incorrect, maybe it's a database that doesn't care about accuracy or something. I couldn't find any. Yeah, it wouldn't make sense, right? Yeah, I couldn't think of a single use case either where i would want that but i thought maybe like a logging situation maybe a sensor where it's like if it shuts off midway like it's really important or like a black box in the airplane it's like hey if you shut off recording uh in the middle last message maybe i really want that last message

you know even if it's partial like i still want want it, whatever you got. But I just couldn't find anyone talking about anything like that. That's interesting. Yeah. So, um, let's see. So the atomicity, they say that's achievable with a log for crash recovery, right? So we've talked about the right ahead logs and all that kind of stuff. So, so basically you're writing to a log before you write the record or right around the same time. And if something fails on the record update

when the system comes back up, then it can go back, read through the log, and say, okay, this is what it's supposed to be, and then redo it from there. I think we talked about that a long time ago. We did. There's also isolation, which is the idea of basically locking

the object to be written making sure that no two writers are trying to write to the object at the same time and sometimes there's some kind of complex subs that allow things like we mentioned incrementing last episode where you might want to have a plus one operator

that can go and read the value and write it in a single operation no way you don't have to worry about you know something happening between that read and write or just kind of does the addition and uh you can imagine some other kind of setups like minus minus uh same kind of thing too i can't think of too many more you would want to do or i guess i'm swapping values so you might want to do like an absurd type operation in a single value. And another thing though, this made me like super appreciate going through the exercise of this book. Again,

putting on the hat of like, you're the person writing the database. Like you're trying to, you know, like if you had to do this from scratch, how might you do it kind of thing, right? If you come from Kafka,

right? When you're from Kafka, right, when you're sizing Kafka, one of the things that you have to consider is the number of topics and the number of partitions that you're going to have on those topics and the number of those that might exist on any one broker in your cluster. And the reason why is because there is overhead for every file handle that Kafka has to manage. And for every partition of every topic, there's two files that are being managed, right? So you could almost think of the same as it, as here, like, so a new thing that I got to thinking about, it was like from the write ahead log perspective, like, well, I never really heard about it written, talked about like this from like a SQL server world, you know, it was always like the write-ahead log,

even in like a Postgres, you know, kind of environment. But like, well, maybe you want a write-ahead log per table. Or, you know, like, would there be an advantage or a disadvantage to when you would or wouldn't want something like that? Like, would you want, you have a file handle for every, for every partition of the table, which it might not be a partition table per se, like not to confuse it with the technical thing. So like the indexes of the table, right. But, you know, and then if you

also had right ahead logs for each individual tables, like how many file handles might be open just just wrap your head around like how many file handles might be opened on like a given instance of sql server or oracle or postgres or db2 or whatever right for db2 no one's there's no file handles open but um no there's file handles open i'm sorry db2 uh there's But you understand what I'm saying, though? Like, so in all of my career up to this date, like leading up to like thinking about like, oh, I want to build out a database cluster. I never concerned myself with like the number of file handles that might be open. And then comes along Kafka. And it's like one of the first things I'm hit with is like, hey, by the way, you need to consider the overhead of the file handle.

And oh, by the way, like in Linux, specifically, like I ran into this specifically for Kafka, there's a U limit that you could set on how many open file handles you will allow the system to have. Because otherwise, like there's a limit that Linux will say like, nope, no more. And you can't open a file. Then you're shot. Yeah. So yeah.

Crazy. I don't know. I don't know if that, if anything like that, he crossed, cross your minds while, uh, you know,

reading this portion of the book. Not specifically. Yep. Man, I'm in the left field. Okay. No no i wouldn't say you're in left field no i was making a joke because i like of course like specifically i didn't assume that you had like that very specific thought like like the file handles right like that that hadn't really entered my mind but there again like when you were setting up kafka you very much were involved with having to know exactly what you need to do there right and and like you like when you were setting up Kafka you very much were involved with having

to know exactly what you need to do there right and and like you said when you're setting up SQL server or Postgres or any of them like you don't you're not bothered with any of that like yeah I mean I guess that another way it's kind of a testament to those products right like the oracles and SQL servers and Postgres and whatever that you know you don't have to think about that but like in a kafka world you you have to be in the nuts and bolts of it to set it up and so things like the number of file handles that you're going to have open come to come into mind they matter i mean it's like put front and center like hey consider this well otherwise it's going to fail right and it's going to fail spectacularly because that's the whole point of kafka is being able to write crazy amounts of data to the thing so yeah i mean it

makes sense um so the the the isolation and all that kind of stuff we were talking about they said these type of operations with the incrementer and all that to keep things in line, they're useful for ensuring good rights when multiple clients are attempting to write the same object concurrently, right? So me, Outlaw, and Jay-Z all trying to write the same record. It says these aren't real transactions. When you're doing a single object, it's not a real transaction. They're kind of lightweight transactions, but unfortunately a lot of database systems out there will call these acid, um, for marketing purposes again, because, Hey, well we support making sure that this single object, right? Happens properly. That's not really acid. So, um, again, it's marketing speak, whatever, but usually when you're talking about a transaction, you're talking about a group of things that all have to happen at the same time and be committed, you know, as a group.

Otherwise removed, right? Like roll back to the original state. You imagine database being like, we fully support ACID transactions. Okay, great. Let me dive into the API and oh, single object transactions. That's a, I mean like you better do that. We've just talked about it. That's kind of a requirement and the most basic requirement of database.

Yeah. Otherwise I just put things in files and call it a day. Right. Like, why do I need your system? Yeah. Yeah. So yeah, about those multi object transactions. So many databases don't have those transactions because they're tough right it's really hard to implement especially across uh different partitions so you know even like past the node level just kind of splitting things up is difficult and it's got a lot of performance implications and uh implications for availability and there's really no technical reason

that distributed transactions aren't possible. It's just a lot of work. And that's something we kind of talked about too with like the database papers and a lot of the ideas that we've talked about kind of coming from the 70s and stuff is just because they were theorized back then, it didn't mean people were actually able to implement.

So a lot of the things that modern databases give us that have come around in the last even 10-20 years, like we mentioned Kafka a lot, are ideas that are old, but it took a long time to really evolve the tools and evolve the systems to where they're capable of actually implementing these global ideas in a way that's practical and usable. Yeah, they get into the details of, Oh, sorry. But yeah,

they get into the details of how, how those distributed tracks, distributed transactions can be made possible in a later chapter. So I do look forward to when we get to that because even non-distributed, like, you know, I still, my mind is still racing with like,

uh, how you'd have to do this. And like the idea that there are these, uh, like blocks of code where you've like, like object locked it, you know, so that, uh, you know, only one thread is going through it at a time. And like, how do you know how many, like all the different files that you had to like put in that same lock block in order to you know write and save them out or not right still crazy that's still a crazy uh thing to me

like because you know they didn't know how many database what the tables were going to be at the time right like uh just i don't know just still uh i guess I guess I'm really growing to appreciate the shoulders that we're standing on that made these things as we get deeper into the subject. Yeah. So they ask a question in the book that I like. Do we even need transactions just as a whole, as an industry? What are the situations in which case we actually truly need them? The best answer,

speaking from a multi-object perspective, is basically anytime there's any sort of dependency between table links and rows of other tables, like foreign keys, for example, where data changes together essentially and there's not a lot of use cases in a relational database so i can imagine where you don't need

those kind of relations yeah what's what's really interesting is they start to talk about in non-relational systems so like a mongo or whatever so typically you don't think about about records relating to other records, right? You store everything in one, in one document, right? But then they bring up what everybody should really think about is, um, even though they're not linking to documents and other tables, a lot of times what they'll do is they'll bury the original document into another one, right? So the easiest one to go with is like a forum post, right? So we all have our user records. Well, in a database, you would just link back to the user ID, right? Like M

Outlaw, J Zach, A Underwood. And when the page is rendered, you go look up the information from that user. Well, in something like Mongo, you're just going to put the user information in the document with the post. And then that way you don't have to go look it up at this time. Well, what they talk about here is, well, even in those database systems, if you update Alan Underwood's information in his profile, that needs to be updated in all those forum posts, right? So that is a relational, like a transactional type thing. And if you don't update that information in all places, then now you're going to have a disconnect from the actual user's information to the forum posts, right? And so that actually should technically potentially be a transaction, right?

You need to update all those instances where that user's profile information exists. Yeah, that's pretty scary. You're getting wrong results. I guess it would depend on the use case, though, because if it if it truly uh was like super important then you probably you might not be using a document database for that particular use case right yeah totally we've talked about search engines a lot of times they don't have precise results because their indexes are kind of i don't want to say sloppy but they're

just not optimized for that and so a lot of times you'll see yeah and you'll see a search result where it's like um showing you uh results one of thousands and you're like well wait how many thousands don't you know it's like nah yeah it's uh you know those numbers fluctuate a little bit and depending on all sorts of situations so well that's the thing about like google even back from the original you know its original days right you know you would do some search and it'd be like page one of you know 999 000 how did you get so many so fast like how do you how do you know okay i want to see 989 000 999 page oh you got a page there hey hey but back to what you said though outlaw and this is this is where you know i i agree and disagree you said

you know people wouldn't use a document database for for something like that right if it was important the relation was important so i don't know that that's true i honestly think that people will start up and be like we're using mongo right like that's that's what we're using we're using mongo they don't care if it's relational or not. They don't care that it's just, Hey, this is, this is the database we're using. And on a similar plane, people that are used to SQL server, Oracle, that kind of stuff. They're like, Hey, we're starting with SQL server, right? Like they don't think about the use case. And I mean, we've, I don't know, maybe we talked about this way back in the past. Like if you're doing a relational database system, let's go back to the e-commerce story that we have. One of the things that is actually

challenging is you have your home address, right? And you order something. Well, in a relational database, you usually have your address attached to your user, right? And then you order something. And at the time that you go look up your, the, you know, it's gotta be shipped. It's going to go look up your information and send it out. Well, that's not actually a great order. Um, e-commerce setup really what you want is, Hey, where do you want this ship to? And that should live with that order. It shouldn't be a relational link to anything else. It should be on that order forever because that is where you said you want that thing to go. And that actually makes more sense for like

a document database, right? Here's your order information nested in it is the address it's going to be sent to nested in it is the, is the order details and all that kind of stuff. Right? And so really what you're talking about is a mixture of a relational database system and an object database make a ton of sense there. But in reality, you know, people are going to use whatever they feel most comfortable with. And that's, that's what kind of stinks. And so it's kind of nice to know that there are these, these, these competing things, especially with transactions that you have to be concerned about. Well, I mean, yes, I agree. In that example, you could do it.

Although, even as you were describing the example, I was thinking of ways that you could easily implement that in a relational database. Totally. You still have an orders table and an order shipping address table where the shipping address is not tied to the uh, you know, the customer's address book per se. It's just, this is the address that's being used for the, to ship the order. Um, so, but, so I think that's, that's different, you know, so I guess the point is that like,

in my mind, you know, if you have a need where it truly, truly matters, then, you know, and depending on the size and speed and whatnot, then, you know, it might be worth going to the database, the relational route. That's not to say that one size fits all is going to work. There was actually, somebody shared an article. I don't know if you guys saw it. I'm trying to remember where I saw it now. I'm pretty sure it was in our Slack where the idea was, you know, if, you know, that old thing about like

if SQL Server is your nail, is your hammer, then everything's a nail and you're going to like, you know. And the article was making the point of like, well, yeah, but is it worth, that's true. And so if you have this one other use case that maybe Kafka would be better, or maybe

a Mongo would be better, or maybe an Elastic would be better. If you have this one other use case, if SQL Server is solving, or Oracle or Postgres, whatever, is solving 99% of your use cases, except for this one, is it really worth bringing on an entirely new technology stack for that one thing? Because even though it might excel at that one thing, you know, the other technology in this case, I was picking on SQL server is a known quantity for all your developer, for all your team. Right. Like everyone knows what its capabilities are, what its pitfalls are, how to deal with errors and everything. So, you know, maybe to your point, if if Mongo is your hammer and, you know, you know how to deal with it and deal with it well, then sure.

You know, if if if there was only like a single use case that, you know, or a limited, very small, limited number of use cases that you're concerned with, it might not be worth bringing on the the bulk of another technology that now, you know, the team has to get up to speed on. And inevitably what will happen is the entire team isn team has to get up to speed on. And inevitably what will happen is the entire team isn't going to get up to speed on it. You're going to have some members of the team that will quietly revolt and be like, I'll never touch that thing. And they'll never bother to, you know, really get up to speed on it. You'll have some that'll like super dive deep into it and figure it out. And then you'll have some that will like just surface level, you know, reluctantly touch it, but won't get super deep. So even then, and that, you know, when you do have problems in that scenario, you're going to have only have those few that, you know, dug super deep into

the problem. So that's the, that's the reality. That is the reality. And that's always, Hey, that's always a question that should be at the back of your head. You shouldn't just willy-nilly add things to your stack for no reason. There has to be a really good, compelling reason. Because there's always overhead and heartache that comes with any new thing that you add. So the adage is the juice has to be worth the squeeze. And that's really what it boils down to. For me, it's about color palette and mascot.

If they've got a cute mascot and they did well with the color palette, boom, I'm adding that to the stack. The logo was sweet. Don't forget about the resume. Resume-driven development. Oh, man. Is that actually a term? There are people that go by it, yeah, for sure. resume, resume driven development. Oh man. Is that actually a term? There are people that go by it. Yeah, for sure.

Oh wow. Yeah. So, uh, I did some Google and try and figure out like some of the more popular document oriented databases, musical databases to figure out like which ones support multi object transactions. And it's funny, like the first article I came upon was pretty good. It was kind of like couch DB. Yeah, it's okay. Uh, Mongo. Yeah, it yeah it's okay you know i'm agreeing to oh okay like yeah something's you know supported it's just kind of dodgy and then uh i get down to fauna db which i don't think we've

talked about before and you know it was great and i realized oh wait this is a this is a blog article published by fauna so okay well let me go look at what how mongo has to say about it guess what this one's okay that one's okay mongo we got it yeah okay all right all right y'all let's see let's see what's going on here so yeah i can't trust uh these sources no you never can um hey so to wrap up the thing with these with these additional things that transactions might actually need to exist for. One is indexes, right? So indexes for tables, those actually have to be updated at the same time that the data in the

table does. So that's important. And then instead, a lot of this stuff can be handled without database transactions, but then you're putting all the onus on the application developer to do this stuff right. And this is hard stuff, right? Like this is, if we talk a lot about, you know, what is the core feature or what is the core purpose of whatever your product is? And if you're spending all your time trying to write all this edge level. Hey, what if this happens? Then you're not focusing on your core product.

And the code becomes really nasty too, right? I mean. Yeah, instead of solving the business problem, you're solving errors. Yeah. And when you think about the kinds of problems that you have, like having problems that screw up your data are kind of like the worst problems to have.

Like, sure, maybe you break the login form and people can't log in and can't check out and you lose a bunch of money. Like, that's bad. It stinks. But if you make it so that all your transactions, like the totals are wrong and you don't know how much people should have been charged, like, whoa, that's a whole different kind of problem that you're going to be dealing with long after you fix the bug. Or maybe the bug is you have that part right, but you don't persist the charging information, but you do persist the shipping. And so you ship out all the inventory, but you never charge. Yeah. You just drop the table. And you've got backups, but the backups are missing a little bit of data.

So it's like you've got to restore the data. And then what do you do about that missing data? Well, I don't know. Backups are always an interesting thing, too, when you get into that situation where it's like, okay, well, because of the age of the backup, we're going to lose all the new stuff. And, you know, the new stuff is like in some kind of weird mixed corruption state. Like, I mean, not corrupt as in you can't read it. Just things aren't in the right places. And we're not exactly sure.

Like some of it we could tell what's what's correct and what's not. But not all of it. Yeah. All bad spots to be. Yeah. You just quit. Go out. You don't come back.

That took a turn. It did. Sorry, everybody. All right. So, Jay-Z, you're giving us a late-night voice here. See, I was about to do it before you did. Yeah, I'm going to do my best Howard Rogan here. I can't.

But what I can do is ask you to leave us a really great review we really love it helps us grow the show and uh especially the more stars the better if i'm being honest uh it's better for everybody it's better for us of course but it's also better for you uh because it makes you feel good to help people out right so hang out hang out with us for a while, you know, listen, enjoy your time, and then go leave us a review because we love it and because you love yourself. I like it. I like it. Jay-Z took a marketing class. Yeah, perfect. Who are you and what do you know about Jay-Z?

All right. So with that said, then, we head into my favorite portion of the show, Survey Says. All right. So what is this? Oh, 203. So, Alan, you are first. And the first question is.

It's at the Lake House. Oh, sorry. That was the last one. Was it? I don't think so. Maybe the one before. Was it? It was like when bad things happen, something like that.

I don't remember. Oh, that was like in a horror movie one, right? Yeah. Yeah. Okay. Now I know where your head's at. All right. Name a reason you might be late for work

forgot the forgot the alarm alarm clock okay yeah alarm clock traffic yeah okay so uh woke up late i'm gonna count like alarms is as waking up late, right? Yeah. Number two answer on the board. Allen taking a lead from the start with 26 points. With traffic's number one. Boom. Jay-Z with traffic, the number one answer, taking away that lead with a commanding 38 points. Oh, come on, man. Yeah.

And Joe, that shifts it over to you. So let's say this one. Name a chore that people put off because they have work the next day. Sure. People don't do because they have work the next day sure people don't do because they have work the next day you'll never hear this out of jay-z none of these would ever be excused from jay-z no he would just do it because i got work tomorrow yeah i mean i i can't think of anything a chore uh all right i'm just gonna uh say something that. Are they chores if you enjoy doing them?

Yeah. That's probably what's going through his head. I only do things that I enjoy. This question doesn't make sense to me. No, I think I just have to not have an answer. Really? Are you still an answer? Anything that I wouldn't do because I've worked the next day.

Oh, sweet. All right. I got one then. Okay. Okay. And chore. And chore. Laundry.

Laundry. All right, Alan. Well, first of all, can we just put like the big goose egg up there? We got to do that. Yeah. Let me go ahead and put this in the official record. Wait, so you're saying you wouldn't do laundry because you have work? Right. You just don't feel like doing it.

Yeah. Yeah. That was the thing. So laundry. I would have to do laundry because you have work? Right. You just don't feel like doing it. Yeah. Yeah. That was the thing. I would have to do laundry because I need clothes to wear. Number four answer on the board. Dang it. This is number one. Six points.

Boom. Oh, come on. JC is still. How about I'm not even going to win by default. This is so awesome. God, I love this part of the show. JC just laid down. I can run over him and still can't think of anything.

You know what? In my next career, I want to be like on a morning zoo, you know, kind of radio program. Okay. So number one answer on the board take out the trash for 35 points are you kidding you wouldn't take out the trash because it's work it's like five minutes dude man i disagree with these people jay-z's like but uh why i like doing that like you get to go outside it's great that's like the highlight of my sundays and wednesday nights wait you only have to take out the trash twice a week yeah man that's life without kids right there

no doubt right multiple times a day over here no doubt it was dishes number two please tell me i'm gonna be really irritated dishes number two answer on the board 30 points i guess i just don't i don't understand this bro why wouldn't you do dishes because you have worked like what you're gonna leave in sync while you go to work did you also don't have kids let's rewind again so you just leave this, like, why wouldn't you do that? Because like, well,

I'm not going to clean these dirty dishes because I got to wake up at eight. Like what? Jay-Z's got a spotless house. Yeah. It really does. That's the thing. This is a done trashes out. Yeah.

And for visitors, there's extra cleaning supplies. As you walk in, like grab, grab a set of of gloves, you know, rubber gloves and some cleaning supplies. You know, anything that you see that you want to clean, have fun. This is, you know, mi casa, su casa. I think, I don't know, like I'm married. If I tried saying like, hey, I'm not going to do laundry because I got work tomorrow. I'm not going to do this today because I got work tomorrow.

You'll lose an eye tomorrow we're talking about today i mean you have 65 respondents that was where we're at so far oh you got time to lean you got time to clean number three answer on the board vacuum 18 points laundry was number four clean the bathroom at number five for four points and dust as the sixth answer for three points i can't believe i picked out of the two i had in my head i picked the one that got me six points instead of 30 30 man all right all right um so jay-z you are no alan you were up next yeah yeah um so i'll let you pick the next question are you ready yep your choices are name a famous wizard name someone you might ask for directions or name something that comes out of clouds. Man, I'm going to do the wizard. I will be really mad when I don't get the number one answer here.

Okay. The Wizard of Oz. Okay. Oz. I like it. Wizard of Oz. Jay-Z, this is your chance to retain your lead. Yeah.

Just know that I picked the number two answer on the board yeah well it's either harry or merlin and i've got to figure out who these people are asking so i'm gonna go with merlin i think it's i think the crowd's gonna skew towards the 1900s okay uh all right merlin number two answer on the board oh man it's 26 points okay now alan i would be really irritated here the wizard of oz is on the board, Alan. But at what position, sir? Well, it's at number five. Come on.

So that's eight points for Alan. Wow. That's ridiculous, man. This is awesome. 64 to 40. JC wins. Wow. Harry Potter was number one.

37 points on the board. Gandalf number three for 11 good doctor strange for number four nine what uh wizard of oz was fifth voldemort oh i wasn't supposed to say it yeah for uh he he was the sixth uh wizard at three and this one if this one had been first alan i would understand your frustration number seven i'm going to mispronounce this name newt scamander oh that isn't that the uh that's the harry potter prequel stuff is that who it is oh it's like the uh fantastical beast yeah that one i think isn't that you're right you're right you're right yeah look at alan with a pop culture reference right i get once about one a year that's it yeah he had he had two two points

man i can't the wizard of oz has wizard in the name yeah how is that like not number one come on man they didn't ask your grandma Oz has wizard in the name. How is that not number one? Come on, man. They didn't ask your grandma. In short, that's the reason. I think you're right. Oh, man. That hurts. But it's true

so uh yeah uh how about we talk about databases now you can wipe away the tears i promised myself i wouldn't cry my grandma would be offended and still very happy all right all right so here we are back back at it so handling errors and aborts jay-z you want to take it i still can't talk yep so uh the good news about asset transactions that fail is that they're easily retried it's because they either happen or they don't right so if they don't happen just try again uh some systems with leaderless replication like we've talked about cassandra before will follow with the best effort kind of basis they're a little bit more loosey-goosey so the database does what it can and if something fails in the middle uh it'll just you know they'll think what it's got which

means uh as we said before it's going to be up to the application to have to deal with that, to kind of take the error message or whatever, and like figure out what it needs to do to fix it or just deal with it being wrong. All right. And not being able to trust your data, which is okay sometimes. But this goes back to your example that you were looking for before the Joe of like where you might take some but not all of it yeah i was thinking more like a um a single note like a single object kind of uh you know level so at the time i was like thinking like literally like sensor data is there something where like i would want to get early if you're recording like you've got an audio recorder right like you know and i don't know this is sunday this is a

silly example but i was just trying to think of like something where you would want to take something like all your changes to the very last second even if it's not necessarily committed yeah i mean audio is going back to your example of the the black box scenario where you know even if you don't have perfect audio of the pilot's microphone you want some of it to be able to hear hear it so from an audio perspective like that that kind of made sense but uh i still haven't thought of like a data use case example where that might be the case but you then you mentioned sensors and i'm like oh i guess i could see that like you know whatever your sensors are maybe they have the ability to, like the fidelity of what they could produce is like, you know, some number of times every second.

And, you know, it's okay if you missed some of those seconds, because in the grand scheme of things, you still might have enough to like get to whatever your need is. But that's still not that single object thing, right? Really what it was talking about was only writing half of a piece of data to make it complete. So instead of skipping seconds, that's more...

Oh yeah, that's true. Yeah, so we're talking about writing an incomplete sensor readout, right? Like, hey, it was supposed to say 78 degrees, but it only wrote seven. That's where... You can still see you know like that's that's where you can still see sensor data like that where it was like oh uh you know the the data point for this time was unintelligible but you know that you received something at that data point so it's logged in the

database yeah that something came in you just it's you know not very good yeah hey so this is funny this is where the book actually called out developers. So talking about this, putting all the burden on application to recover from error on failure, they basically said developers like to only think about the happy path, right? So we're going to write our code and assume that everything always works. Well, there's a big problem with that because as soon as something goes wrong, what are you supposed to do? What is the application supposed to do to recover? And this is actually, you can spend a lot of time as a developer trying to do this right. And I think that's why a lot of people just go the happy path, right? Like, oh, the database will be up and

working. I don't need to worry about it, right? Or I know this will be up and working. I don't need to worry about it. Right. Or I know this will be up and working. I don't need to worry about it. Well, then when it doesn't work, um, that's when things fall apart. I mean, it's kind of like the nature of how we, how we do our jobs though. Right? Like you don't start, like if I asked you to write like a command line utility to do something, right. You're going to start on the high path of like, oh, what's the thing that I'm trying to get it to do? Okay, I'm going to start down that path of writing it. You're not going to write code first and be like, well, what happens if the user input this value instead? Right.

Yeah, I mean, they're not wrong in calling out developers for that, but you're also not wrong in that you don't go for the most complex edge cases to start out because you never get your stuff done, the things that you actually need to do. Yeah. Is that what you said, Yagni? Yeah, and just like we talked about gold plating,

imagine you're trying to write some little shell script to do something every 15 minutes and then you're so wrapped around thinking like, well, what if it fails? What if the memory fails? What if I can't trust the data from it? And next thing you know, your one-line shell script is like a thousand lines of application code. And it's like super brittle and you can't make changes to it. Unless you did it right, in which case it would look like the spring implementation of FizzBuzz.

No doubt. No doubt. Hey, so then they also called out there's a lot of so databases a lot of times are built with transactions in mind right to do these things for you they called out that there are a lot of um like object relational mapping databases or or orms out there that don't really do proud by transactions, meaning that if something goes wrong in the ORM, rather than building in the ability to do a retry, which is typically what you'd want in a failed transaction, something to go back and try it again, it just basically blows up and bubbles the air all the way up.

And they said that that's sort of not in the vein of what a transaction should be. So if you're building an ORM on top of a database, it should probably have in some sort of failure triggers and things that you can hook into. So they called out Rails Active Record and Django specifically. Okay. but I kind of took issue with this part. I'm with the author in the case that I want that as a capability to where I can say maybe as a parameter when I try to commit the transaction, I'm like, hey, here's a number of retries that I'm willing to accept. What I don't want though is for every transaction

I ever do to automatically get some kind of retry logic added onto it because there might be cases where I legitimately want that error, the first error bubbled back up to me and I might need to re-evaluate what decisions I'm going to make. Do something differently. Don't want it to be every time automatic without my say so. Yeah, I can agree with that. I thought it was interesting. I don't know that I'd call it out either because it should be easy enough to trap in your application and throw it into a retry loop or something.

But it was interesting. Well, I can kind of understand, though, where like you might want the, you know, just playing devil's advocate with myself, though. Like, OK, so you said trap it in the application, but that means you've now gone full round trip. If there's the ability to like as part of that commit to tell the database like, hey, you know, this assumes that now now we're not at the ORM level. Now we're assuming that the database itself has a retry capability as part of the commit if the first one didn't succeed for some reason. Then maybe you can avoid that round trip of doing it if you can tell the database to do the retry. You know what I'm saying? Do we know any databases that have a retry in it?

I might be just thinking in a nice, clean, optimal world yeah when you said that i was like i don't know that i mean maybe it does now i'm gonna go look anyways all right so next thing that they hit on is even acid transactions aren't perfect right um we we hit on this a little bit in the last episode. So they say, hey, this goes back to what Outlaw was talking about earlier. What if a transaction actually succeeded, but the notification to the client got interrupted and now the application thinks it needs to try again? You might end up actually writing duplicates. So this is this is exactly what he said.

You submitted some things to the database. They completed successfully. So it wrote those records out, but then it fails before it can send back the notification to your application that everything worked. So your application thinks it failed. So now in your application logic, you might have it go into a retry thing. Well, if this wasn't an item potent type, right? Like in the situation where you're writing an order and some order details, um, you might duplicate that data. And, and actually in this case, it's worse than duplication because now you have a new order with new order details

that mirror the other ones just because the database wasn't able to send the signal back to the application that it succeeded. So you got to be careful about stuff like that. They even talked about what if the error is due to overload? And this is basically where it'll constantly error. We've talked about it at work as something like a poison pill sometimes but it'll get stuck into a state where it's just constantly retrying retrying retrying and it won't it won't ever stop so basically it's just it's killing the system right it's almost like ddossing um your system to a certain degree um you have to be careful about stuff like that because it can get stuck into a state like that

and you need to be able to exit out of that. And also sometimes retrying isn't going to help at all. Like if you've got some sort of network error or something, all the retries in the world aren't going to work. And so you're just kind of wasting your time. But if you had your application, see the type of error, maybe it would know, oh, the K in the database is down. Or maybe you're in a situation where like sometimes

it's not normal for databases to be unavailable or network to be unavailable. I know a guy who works on cruise ships, and sometimes they go, like you mentioned, like an Alaskan cruise or something. They get far enough away from their internet connection that they're not able to communicate back. And so sometimes their data centers are just unavailable for hours and hours and hours. And that's totally normal and okay.

And they plan for that. But you can imagine if you had these systems that are just like trying to retry constantly and just kind of spending their cycles, wasting time. And it's actually not that big of a deal. Yeah. So I did some quick searching. I couldn't find. I think you're right. I couldn't find one where like there was the ability to pass in a thing to the database as a parameter to say,

hey, just in case, here's a number of retries I'm willing to accept. There were examples of where you could manually do it, even at the database layer. And then I did stumble across some things like with Unity Framework, for example, where it could handle the retry logic. But that seems crazy to me. There's not a way at all. I mean, I guess I never thought about it. I never cared. But I'm surprised that it doesn't exist to where you could avoid the round trip.

I mean, you did call out that there are certain scenarios like a retry isn't worth even bothering with because like you know if it's a constraint issue that you're not adhering to then it doesn't matter like no bother no point in retrying yeah that was actually an additional one on top of it right so what what jay-z had talked about was like a network error but what you just said is yeah if you kept retrying to do the same thing that you were trying to do and there's a constraint in the database that says you can't do it, you can try it a billion times and it's going to fail every single time. Right. So this this this actually steps off into something a little bit different, at least in my mind. Like a lot of times when you see people do exception handling, they'll just, they'll have a try catch and they'll just catch on an exception, right? Instead of a specific exception, like,

you know, database can't be reached exception or whatever. This is actually a time when it makes sense to have an error trapping so that, you know, Hey, did I get a, did I get a constraint exception from the database? Because if I did, then I know that this thing's never going to work again. I should not retry. If I got a network error, well, then maybe I do want to retry, but maybe I'll do it with a back off of 10 seconds, right? Something like that. So this is where like having multiple levels of exception tries or catches will make a lot of sense for you, right? Especially if it's something that you know happens a lot. You're not going to go program this stuff off the bat like what you said earlier.

Like it's a bunch of agony garbage. But if you see these things happen, then it makes sense you log it out to whatever your system of logging is so that you could then in some kind of a dashboard expose a metric to show like, hey, these are the number of deadlocks that we face or whatever our retry cases might be. And maybe we need to reevaluate how we're tying together certain pieces of data because look at our dead deadlock rate here for example right so back to the devops handbook there well no the the google sre i was i was thinking of the google sre book more specifically um just trying to like i'm getting an error what's the rate of the error like how often is this a problem because if you simply did the retry of that, of that commit,

um, in a loop, you know, on, on your, and maybe like, maybe you have, Hey, I'm going to retry it three times. And more often than not, you know, you're getting that error and it succeeds either on the second or third time. And so you never, ever bubble up anything. But if you, you know, going back to the Google SRE approach of exposing a dashboard for that or a metric for that, then, you know, you might find out that, you know, there are a bunch of quiet failures, you know, that eventually succeed. But, you know, it might be a sign of you need to rethink how you have the data structured to avoid those pitfalls. I don't know. Crazy idea. All right. And then touching on another thing that Outlaw brought up earlier is, hey, you know, there might be situations where your transactions trigger other actions like an email or an SMS message or something like that. You want to make sure that you're not just firing those things off willy nilly, right? If you're in a retry loop, you don't want to spam the people, you know, hey, this thing failed five times. So you got

five emails, right? Like you don't want to do that. Yeah, this was I guess the example that I was referring to earlier with the email example. Yep. And then the last thing that they talk about is when you're dealing with multiple systems, you may want to do something that's called a two-phase commit. And again, that's coming up. I think we talked about earlier, there's an implementation that's coming up later in the book. So is this.

For distributed transactions. And that's one of these things is this two-phase commit. So we'll be talking about that in a later episode. Would that be like the wedding commit? That's where like both sides say I do. Is that what a two-phase commit is? It might be. We might be able to bring it back to that later.

Stay tuned to find out. All right. to that later. Stay tuned to find out. Alright, well, with that, we'll have links to the resources that we found that were helpful, resources we like, a portion of the show notes. Obviously this book will be

in that section. And with that, we head into Alan's favorite portion of the show. It's the tip of the week. It's certainly not Survey Says because those people are always wrong. That's right. So have you all ever heard of ManyThing? I have not. I've heard of ManyThings before.

Which thing are you referring to? So I'm talking about an app called ManyThing that you can install on your favorite devices. You know, you literally wrote this in the show notes, and I just assumed that it was a placeholder because you had many things that you were going to share as your tip of the week. Nope. This is literally an app called ManyThing. So let me give you a use case here before I tell you what it does. Let's say you've got more than one pet, and one of those pets is misbehaving, but you're not sure which one.

All right. So, uh, I don't know, say one is like keeps flipping the food bowl or maybe one is, uh, uh, using the bathroom inappropriately.

Right. And, uh, you're not sure which one, you know, to get the appropriate vet care or whatever. Uh, you can go buy a camera,

like a nanny cam or something, or ring sales cameras. you know, it's, it's expensive and takes shipping shipping time and if you've got a problem going on like sometimes you want to just set it up like tonight and figure out which little rascal's doing you wrong well mini thing is an app that lets you install it on older devices say like old phones or tablets that you might have uh sitting around and use it as basically a security camera what you do is you put it on your old phone set it up hit record on the camera it's linked and it's got a motion sensor in the app so basically it starts streaming changes up to the cloud whenever it sees you know motion and you can configure all sorts of different ways and you

can use another device like a laptop or a phone or whatever to go and log into your account and see your footage. And so it's perfect for setting up and spying on your pets kind of temporarily without having to invest in an expensive camera that takes seven days to get here via Prime. Thanks, Prime. I love the subtle jab out of nowhere. Yeah. i love the subtle jab like out of nowhere yeah i mean i remember days that's on prime that's how you know i'm old it's like i remember when prime was two days shipping yeah not new longer but okay i think it's nice uh especially for the temporary stuff because you can sign up for basically like one month uh catch your little rascal and then do what you need to and then there you go done okay let's get on to the important uh part of this though what was the little rascal doing oh uh there is

one little rascal that was going to the bathroom outside litter box ah yeah and now i know which one so now clearly you've put like a little shocker down there so jump into the box yeah i'm curious if you hold them appropriately or you know take them to the vet so do like imagine if you got two cats both are terrible and one of them is having you know like issues right and it's probably because they're having an infection or, you know, they can have like crystals or stuff like that. You know, they're not doing it for fun. They've probably got something wrong with them, but you don't know which one to take, right? And it's traumatic for everyone involved, you know, with certain cats to get them to the vet. So that was interesting because when you said take them to the vet, I was like, what are they going to punish them with?

No, they're not punishing him. Wow. Wow, Alan. Right? I mean, that's where my head went. No punishment. Okay. Fixing.

Fixing the cat. Yes. Helping the cat. Yes, helping the cat. Yeah. I know what you mean because it could be like kidney issues or bladder issues that the cat's suffering from. And so they have trouble like getting into position let's say yeah right and you can imagine you take both them in the

vets like actually both look okay we can do some sort of uh procedure to kind of cut them open and see and you know do that remember like well which one is it and alan's like yes yeah everybody this is pretty brilliant man like using your leftover devices because we know we got a drawer full of phones and garbage laying around it's really great for the whole like oh i think there's something i want to record wouldn't be nice if i could do that tonight uh you know it's it's not something you'd want to set up permanently you don't want to like old iphone 4 like out there you know strapped to your house trying to act like a trail cam or whatever right it's really great for those little things like i just need to know something happens in the next 24 hours whatever that's pretty cool

yeah i was i was honestly as you were uh going through this i was thinking of like just when i was going to clear out my drawer of old ios devices now i have a reason to keep them around yeah never get rid of those yeah keep the keep the drawer um yeah so it's available on both uh apple and google's app stores uh and the many things stands for monitor anything i think that's important i don't know yeah i didn't hear you say that nope i didn't know that all right so uh let's get into my tip of the week's chapter one. You know what? Inevitably, this is the way it works out. Like days ago, I had a tip that I had in mind and then I forgot it. And then as we get closer, I'm like, oh, I got to come up with a tip of the week. And then I'm like, okay, well, I guess this one will do.

And then eventually, like later, I'll be like, oh, this would be another good one too. I'll add that one. And I still never remembered the original one. And then eventually, like later, I'll be like, oh, this would be another good one, too. I'll add that one. And I still never remembered the original one. And then eventually I do. And then I'm like, oh, well, now I have. OK, so here we go. So I'll make these simple. So the first one is the Linux Foundation. I don't know that we've ever talked about this in terms of a learning resource.

Right. But we've definitely talked about other things, Educative, Pluralsight, Udacity, whatever, you know, Udemy, a whole bunch of other. But did you know that the Linux Foundation actually has a great collection of courses available? And some are free, some are not. So it depends. But, you know, you can find some really good training material out there. And, you know, like I said, some of it, some of it's free. There's an introduction to Kubernetes, totally free. Then, you know, you can get into some that are not free. So Kubernetes for developers, $300 or Kubernetes fundamentals for the the exam that includes the exam.

And that one's $5.95. But the point is, you know, introduction to serverless on Kubernetes, $0. Introduction to service mesh with Linkerd, $0. So there's plenty of, you know, free courses out there, a lot of good stuff. And also maybe topics that you are interested in. And, uh, you know, it's the Linux foundation. So even if you were to like pay, uh, you know, for a course, uh, you know, it's going to a good cause. So, um, so that there's that one. Then, um, this one, I don't know that we've talked about this one, something like this before, but, uh, we, we've definitely talked about our love of

Kubernetes and all things Kubernetes. And Kubernetes actually has a list of recommended labels that they suggest you add to your pod. And it was interesting the way, you know, some of them, so like one of them was part of. So, you know, you might have a component. So component was another one of them, right? So the component is database and part of might be WordPress in their example, right? And where I specifically found this interesting was that the way they were using version versus how I recently implemented our use of version in a different use case. And I was like, huh, I guess could you maybe argue I'm doing it wrong or maybe I'm doing it okay or maybe I should have done something else.

I'm still kind of on the fence about that. But at any rate, so I'll have a link to that of the recommended labels that Kubernetes suggests you have on your pods. That's very cool. Yeah. And then the one that I had wanted to talk about in more detail was basically our continued love of Docker and all the crazy things that you can do with it. And so I had this use case where, um, I have been, there's this one, uh, there's, uh, this is for music. So, you know, I'm a nerd out from it.

Uh, but anyway, I was trying to like learn this one particular song and I can't find anything about the song, uh, anywhere like written down in written form, uh, in regardless of, you know, the format. And so, uh, you know, except for like super basic stuff, but none of the detail that you really want to, you know, um, and so, uh, there was this one video though, where somebody was playing something and, you know, they're, but they're playing it at speed and it's not like a, Hey, here's how to play it kind of less. And it's just more like, Hey, look at what I'm doing. And, uh, I wanted to be able to like go scrub through it. Like it changed the speed and zoom in and see like, wait, what are they doing there? Like, cause it's really fast. There's a lot of gain on it, you know, a lot of distortion. So it's kind of hard to make it out. Um, and so at any

rate, I eventually decided, you know what, YouTube, uh, this isn't working out for me anymore. I want to just download this thing. And so there's this other, there's this great package out there on GitHub called YouTube DL, and I'll have a link to that. But what I ended up doing was putting together a Docker image that runs this as the entry point. And you could just give it a URL and boom, it just starts downloading whatever you want. And the cool thing where I originally had discovered this thing, by the way, was that a friend had given me a video in Vimeo that I wanted to add to a YouTube playlist. And I'm not a Vimeo Pro customer, so I couldn't get to the video. And it was behind a password. So downloading the original video was like super difficult. YouTube DL, this package here, YouTube and Vimeo, there's a whole bunch of different sources of

places where it can go. And one of the things that it can handle are passwords. Oh, nice. So, so it'll, it'll do that authentication and like, as it's downloading it, it'll download it in chunks. If it needs to run it through FFmpeg, uh, for any, any issues there. It will do that as part of it, but then it'll ultimately rejoin the parts that it's downloaded and clean up after itself. But you can, like, short term, like if you wanted to keep those intermediate parts, you can too. So in fact, it just dawned on me too. Another thing that I could provide would be the, uh, I can provide a link to the Docker hub or the Docker,

uh, image that I created. Cause of course I put it on Docker hub cause everybody wanted it. So I was like, yeah, they're calling at the door. Do you remember, uh,

the big scandal around YouTube DL, like maybe a year or two ago, three, eight. I remember, I know that they, they for a while took away youtube support and then they added it back but that's as much as i know of it so it was getting removed left and right uh there were takedown like youtube had issued some sort of takedown notice so it disappeared from github but people had backups of it so they're putting it up under different names and people started writing like tools and websites that would like help you find like

another another place to download the package so it's like this war between like google and takedown notices and like just individuals putting this up in other places so it's good to see that it's i don't know if it's good or not but it's interesting to see that it's uh still on github i guess it violates the terms and conditions to download videos from YouTube. So I'll put a note in the show notes about speaking with your legal team before you download any videos. But we'll have instructions on how to do that as well. Well, YouTube has the ability to download. They offer the ability if you are like a YouTube subscriber.

But not automated i think i think it's you're not allowed to use like a bot or something with that kind of stuff i mean that that's a lot of the license agreements with a lot of these things out there like yeah you can do it but you can't have a program that does it. Well, okay, fine then. I'll take back my tip of the week, and I had two previous ones, and you can use one of those. So... That's content. I did look it up,

and you should speak with your legal team about it, but certain usages may be legal or not legal depending on your country and state and use case. So... Those are the people I'm talking to. Yeah. All right. Use at your own peril.

What were those countries again? Remind me, I forgot. The ones that I'm talking to, Jay-Z? Legal team has advised me to no longer speak on this matter or give any sort of specifics or legal advice. Amazing. If you need a legal team, you can click on our link at cuttingblocks.net and use our affiliate code to sign up for legal advice.

Of course he's lying, but that's fine. I mean, I'm going to look for that now. All right. So still a pretty cool tip. And the fact that you can dockerize anything. For what it's worth, I'm very similar to Outlaw in this regard. If I can avoid it, I install nothing on my computer. I Docker run everything and try and do it that way. I mean, that's the real that. Oh, that was the real tip there. It was to just Dockerize

everything so that you don't have to like bother having a local copy of Python and FFmpeg and yeah, man, stuff like that. So much better. All right. So mine, I borrowed happily from our Slack channel and I didn't actually take note of who put this up here. I can look real quick. Okay. So it was Nathan V shared this and it is, it's called Marp for visual studio code. And you either know this or you don't so marp m-a-r-p is the markdown presentation um and it allows you basically to create slide decks using markdown and there is a plugin for visual studio code so if you're somebody that loves markdown which a lot of developers do and you are creating slide decks and you like doing presentations you might actually

want to take a look at this thing um it looks like a really interesting uh plug-in for visual studio code to be able to do your your slide decks right in visual studio code you ever feel like we're gone we've gone full circle like do you remember this is this is going to show uh an age but you remember like the old school text editors like the word processor type editors where like if you wanted something in bold there was a syntax that you would have to put before and after whatever you wanted in bold or some like almost like it was in mark like a html kind of markdown kind of thing except this is predates that by a long shot gml right i think is what it used to be called.

But yeah. I thought it was predated even that. I mean, talking about like from the 80s or something where like, you know, to bold something or to put it into italics, there was a command sequence that when printed and you had to do it, but you would see it in the raw. Right. You know, and we were like, you know what?

This is just too noisy. I don't want that. i just want my word processor to just show it to me which eventually they did do and then they got even more advanced with like word now we're saying like no no i want to see the raw guts of what it's doing so like take me back and uh you know now we've markdown to solve that full circle yeah always the best totally yes yeah all right well uh with that wait i'm alan you have like 17 more tips there right like oh i ran out of wine i totally forgot i mean i might have figured out something useful to do this week but i can't remember any of them you know what i'll tell you what alan i'm gonna be nice you can have that youtube one since that one was like uh legally you know questionable oh yeah there we go like it so alan has his two tips of the week uh you know i have mine in okay so i think we're all good there so uh with that

hey hold oh no dad jokes has the well run dry? Did I? No. No, definitely not. I just hadn't even thought of any to share. Now that you put me on the spot. You went very robotic when I asked you. I hadn't really thought of any to share. It just hadn't dawned on me. Yeah.

No, I definitely have some at the ready, but it hadn't dawned on me that I needed to do it. So I kept forgetting. So at any rate, how about this one? We'll end on this one. Why did the chicken cross the road? Oh, geez. I can't think of a reason. To get to the dummy's house

okay knock knock is somebody here hold on hold on someone's in the door yeah I'm coming I'm coming oh oh oh geez you got me alright so with that Oh, geez. You got me.

All right. So with that, subscribe to us on iTunes, Spotify, Stitcher, wherever you like to find your podcast. We hope we're there. And be sure to leave us a review, as Joe had so eloquently asked for earlier in the show. You can find some helpful links at www.codingblocks.net slash review. And this one's going out across the wire to all the coding blocks listeners out there.

While you're up there at codingblocks.net, make sure you check out our show notes, examples, discussions, and much more. And make sure you leave your feedback questions and rants to our Slack channel at codingblocks.net slash slack. You're muted. You're totally muted. Oh, I did a great job, too. I'm sorry.

I just unmuted. Okay, sorry. We got Howard Rogan here. Make sure to follow us on Twitter at CodingBlocks or head over to codingblocks.net and find all our social links at the top of the page. It was better the first time, but how many packs a day do you smoke as many as it takes