Utilizing Tech - Season 7: AI Data Infrastructure Presented by Solidigm - 11: Is Kubeflow the Real Deal? Featuring @AponteAnalytics and @DPBrinkm

Episode Date: November 3, 2020

David Aponte and Demetrios Brinkmann discuss the future of Kubeflow with Stephen Foskett and Andy Thurai. Kubeflow is getting a lot of attention, but contributions and community seem to be lagging. Is this really the future of machine learning or just another dead-end open source project? Will product vendors pile on top, redirect the project, or kill development? The open source community tends to gravitate to quality projects, and it will develop Kubeflow (or not) based on the usefulness of the solution. Episode Hosts and Guests David Aponte, Machine Learning Engineer. Find David online at LinkedIn.com/in/AponteAnalytics and on Twitter at @AponteAnalytics Demetrios Brinkmann, Community Coordinator for @MLOpsCommunity. Find Demetrios online at LinkedIn.com/in/DPBrinkm and on Twitter at @DPBrinkm Andy Thurai, technology influencer and thought leader. Find Andy’s content at theFieldCTO.com and on Twitter at @AndyThurai Stephen Foskett, publisher of Gestalt IT and organizer of Tech Field Day. Find Stephen’s writing at GestaltIT.com and on Twitter at @SFoskett Date: 11/3/2020 Tags: @SFoskett, @AndyThurai, @AponteAnalytics, @DPBrinkm, @MLOpsCommunity

Welcome to Utilizing AI, the podcast about enterprise applications for machine learning, deep learning, and other artificial intelligence topics. Each episode brings experts in enterprise infrastructure and artificial intelligence together to discuss applications of AI in today's data center. Today we're discussing Kubeflow, but first let's meet our guests. Hi, my name is David Aponte and I'm a machine learning engineer. You could find me on LinkedIn at David Aponte and you could also reach out to me via the MLOps community.

Hello, what's up everyone? I am Demetrios Brinkman and I am the MLOps community organizer. The best way to get in touch with me is through the MLOps community slack channel I'm Stephen Foskett organizer of tech field day and publisher of gestalt IT including the AI field day that's upcoming you can find me on twitter at s Foskett and you can find my writing at gestaltit.com and now let's meet our co-host Andy Thry I am Andy Thry. Hi, I'm Andy Thurai, founder and principal at thefieldcto.com. You can follow me at Andy Thurai or thefieldcto.com. So we were just discussing something over at the fantastic MLOps community that you run, Demetrios. Over there, there's a questions and answers thread

that's 50 answers long now about Kubeflow. And basically the question is, is this thing the real deal? Is this truly the future? Is this open source project gonna rule them all? Or is it kind of petering out? There's a lot of questions. There's a lot of things that come into that, a lot of baggage with that.

So maybe you can introduce it just quickly and just tell us what's up with the thread and then we can kind of get into the question. Yeah, so I think that there has been some talk around Kubeflow and where it's going and if it is dying, we could say. And I think there are other pieces of this puzzle that there's a lot of people that are contributing. if you're looking at the actual github the cube flow project seems to be like there are less and less contributions but that we have been talking about and on the thread I'll let David get into a bit more because cube flow is so many different pieces that are all kind of Frankenstein together.

Now we're seeing that the contributions are being put in the different sectors of Kubeflow, the different pieces of Kubeflow. Yeah, well, that's, you know, those of us in this space have watched open source projects, sort of like master of master projects like this for a long time. Like, I mean, OpenStack is the one that immediately springs to mind, which became sort of like Frankenstein's monster of open source projects. So, you know, what do you think, David? Is this another OpenStack or is this the real deal? Yeah. So for anyone listening that's not that familiar with Kubeflow, let me refresh your memory. Kubeflow is essentially an ecosystem of tools for machine learning on Kubernetes. So it's built to be native to

Kubernetes, native to the cloud. And it at the moment is most people think of it as one thing, but it really is actually a lot of smaller projects. Things like KF Serving, Selden, Katib. There's a lot of Qflow pipelines. It's actually a lot of different things. And some of them are actually very separate. And because of that, there's been some, I guess, criticisms around the lack of cohesion as a product. And I guess what started all of this was, I'm not going to mention the person's name just out of respect for them, but they basically asked this question, if Kubeflow is not going anywhere. And they asked because they noticed that contributor activity was basically zero, right? It's like decreased drastically. So if anyone's interested, go to github.com, check out Kubeflow.

And you could see in the analysis section exactly like how the trend has really decreased over time. And there was also this other interesting part where one of the, I forget his last name, but David, who was once leading the project now left and is working at Microsoft. So some people asked the question, is the vision loss, is the thing that was going to unify these things no longer there? And then a lot of people were saying that there's a lot, just a lot, a lot of companies who say that they have tried it, but ended up coming to the conclusion that it makes no sense to them. And so there was a lot of interesting conversation around that, mostly by other vendors in the field that are working on products relevant to this sort of conversation. But yeah, it definitely brought up a lot of interesting questions. So hopefully it

gives you an idea of what the thread was bringing up. Let's start out with that itself. So it started with Google, as you said, by Google to sort of try to make things easier to deploy ML models on their cloud platform. So does it mean that A, first of all, is it tethered to Google Cloud very closely? B, not just as cloud as a platform, but also is it tethered very close to some of the Google's tools, machine learning tools like TensorFlow and whatnot? Or can it be, is it truly, you know, that you could use it across the platform across the tools to use that if not then there's

there's no value in it for any company they they say at least in the community that it is a bit harder it is cloud agnostic and you can use it with anything in theory but it is not so easy in practice and i know that i've seen in the Kubeflow community, people that are asking stuff about AWS and how do you put it up on AWS or Azure. And another friend of mine just told me that the new release, the 1.1 release made him feel like they were going even closer to GCP and making it even easier for you to use on GCP because it has integration with config connectors from GCP. So that is some of the complaints that I've heard, some of the different things that they're saying. But in theory, it is completely open and you're able to use it with anything.

It's not tied just to TensorFlow. And I think David was going to mention some stuff about how you can use the different PyCharm or PySpark, whatever you want to use on it. Before David jumps in, let me also throw in a quirk in there. I agree with you on that, you know, the integration connectors will make the deployment easier. But if that's all the tetanus that they have forcing people to use it, that's actually, A, important and useful to the people so you could, you know, get it out faster. B, then it will motivate other companies to build integration, whether it's AWS or Azure, which is really good.

So if that's what they have, that's kind of, you know, down thing, I would say no, it's a positive thing. You mentioned first the question of whether it can work across multiple cloud platforms, and yeah, in theory it can. You know, at my company, we use AWS, I've been able in AI, and we use Qflow as well, we have it deployed. Definitely has its challenges, we've dealt with that. We have it deployed, definitely has its challenges. We've dealt with that.

There's a bit of a learning curve there. And we have a team dedicated to that. We have a machine learning infrastructure team that's dedicated to deploying it, maintaining it, monitoring it, and essentially building abstractions that make it easier for data scientists and engineers to build models and experiment

and maintain reproducibility and all of these good MLOps practices. But it's, you know, yeah, in terms of like compatibility, well, what I'm noticing is that a lot of companies, especially the ones that are contributing to it, such as Bloomberg, and I'm trying, I can't think of any other names right now, but they're utilizing the parts that make the most sense to them. So for example, at Bloomberg, it's mostly KF serving. And for those of you who are less familiar, it's a component of Qflow that allows you to deploy your model as a microservice that's serverless. So for those of you less familiar with serverless, it could scale up and scale down based on the needs or the load. And there's some really nice things about that,

you know, and it's actually made deployments a lot smoother, you can do some more involved things, such as, you know, route traffic to different revisions, meaning, let's say you can do some canary deployments. And if you have that system, you can basically do a B testing with that. There's even a lot of collaboration with other tools that allow for interpretability. So you could add things like Alibi Explainer. So it's just, it's this great tool for deployment. It's open source. We use it ourselves as well. And some companies have basically contributed to that because it was in line with what they were doing. And so it made the most sense to them, but they, I noticed that they also don't use all components of the Qflow system. So I think there is some validity to the criticisms around it

with respect to just kind of not all being completely cohesive yet. And I don't know, maybe this is just me and maybe it's sort of an aside, but I don't see that as a problem. I feel like there's been other, the question that that's even asked in the beginning about other open source projects and it being sort of a Frankenstein I think that you know like I'm optimistic about free markets and optimistic about about open source I'm optimistic about the community and I do think that over time like the the best solutions will work themselves up to the top and I like Qflow in particular because it's open source and that does make a difference. The

barrier to entry is a little bit easier than some of these enterprise products. And so for that reason, even though right now there's not a lot of activity, I'm still optimistic that companies still have the freedom to make their own infrastructure based off the pieces of Qflow if they wanted to. And that's how we're using it. That's how actually I've seen a lot of other companies use it, smaller ones. And let me not overemphasize how awesome it is. Definitely there are some challenges. And that's part of what the problem is. And I think that what I've heard from a contributor is that there's a need for other companies to invest in Kubeflow as well.

And I think this goes with a lot of open source projects. It needs backing from teams that have mature engineering and the availability to make this what it needs to be. It won't just happen on its own, but, you know, people have always been, have always questioned open source, you know, it's just, it's always like that, you know, but a lot of these projects end up, you know, lasting because of the flexibility, which I think is one of the biggest benefits, but also may be a downfall. So Qflow is probably exploded in the scene a little while ago, but now it kind of dying down a little slowly or not much activity. But about six months, even a year ago, they were facing a lot of, because it almost sounded

like it was not ready for production level. And there were a lot of influx of GitHub issues, and then people have to triage that and then assign it to appropriate SME to fix it. And then at times they were not even responding. And if you go and ask a question in their Slack form, there was nobody responding to it. So are they really fixing it? Is the community still coherent together? Is it going places? What's happening? I think this speaks to the point that David was making earlier is how these different clicks are forming or the different subsets are forming.

And they're really going hard in some directions or there's a lot of energy being put into certain parts of cube flow and then you have other parts that maybe it's not that they're forgotten about but if you look at it as a whole and it's not the most important problem for these big companies then it could get forgotten about. And one thing that I have pondered with David actually a bunch is, will Kubeflow become the de facto tool for MLOps? And then will everything else, every other vendor be around that? Sort of like what you have in the kubernetes space how kubernetes is the

de facto open source tool there but then you have a large ecosystem that's on top of that and there is a lot of vendors that are doing things with kubernetes and so i wonder about that. David says, no, I am still open. I think that we got to this point like, yeah, maybe if you can get one-click deploy, but you also have so many different pieces that you would want to use in this, as David mentioned. And I think that there's a lot that is still unknown, but it's going to be a great two years to see what unfolds. Well, on that point, I think that what we've seen always in open source projects is that companies are going to do what companies are going to do. I mean, I remember back in the 90s, all the tearing of hair over IBM is going to take over Linux, you know, things like that.

You know, I mean, as soon as companies get involved with open source development, well, number one, they're going to start trying to direct it or drive it in the direction that serves their needs. You know, whether that's, you know, oh, we're going to make, you know, Kubeflow support, you know, AWS or, you know, a special, you know, infrastructure in, you know, this or that cloud, or, you know, we're going to, you know, kind of take it and build on it and have that be the foundation for our for-profit product or whatever. And then immediately also you get all this kvetching by people in the community saying, oh no, they're going to take it over. They're

going to kill it. But, you know, I think at the end of the day, what we're going to see is just sort of, well, either the development's going to happen or it's not day what we're going to see is just sort of well either the development's going to happen or it's not the community is going to develop or it's not and frankly that's not up to the founders of the project that's not up to the companies that's up to the people and you know the amount of conversation that we're seeing in the ml ops community of this suggests that maybe you know maybe this thing is you know maybe it does have legs that's a good point there's a lot of, there's always a lot of chat about it. Why, if it's like, you know, if it's, if it's that bad and if it has no potential, why are companies building on top of it? Why are companies using it? And they are,

that's the thing, you know, there are some large companies handing large scale and there's even companies that are, are meeting the limits of what you could even do with things like Kubernetes and having to change that. This is like an ongoing issue in just software development. Like you're, you're always going to be like, you're going to always reach a point where the tools that are out there are not enough. And so you need to develop your own layers of abstraction on top of that to make it work. And I, like you said, that's not really, no one company does that or create and can do that. It's like, you know, lots of things that are you know that that are created naturally like you know like languages or money or something like that i think maybe not exactly the same way but what i'm trying to say is that the best solutions end up actually

winning over time that's what i think um and yeah like if if the barrier you know to and if the like the entry to use some of these tools remains difficult like for example one of the reasons why I actually wouldn't recommend a smaller company just starting to utilize ML to use Kubeflow is because you need a team of engineers who are comfortable working on a Kubernetes stack and you need them to be able to know how to debug things in production and there's that's not you know easy always there's a there's a high learning curve you know or steep learning curve with respect to some of those things. So it doesn't always make sense for everybody.

So if there was the ability to make it super easy, then that would even make it more attractive, you know? And, you know, you could see from the, you, you mentioned about like kind of taking over, you know, that one taking over them all. The reason why I'm not so convinced by that is that even Google itself, like if you read the way that they're talking about ML infrastructure and how they build it, they don't always like it kind of hints at certain parts of Qflow, like a lot of the pipelining aspect. But they also really promote their own solutions like the AI platform. And so that's not Qflow.

And so it's not even if even though they've had a big influence on it and it's most commonly associated with Google, it's actually not just Google anymore. It's becoming a little like, like I think because everyone is kind of building their own solutions around what's important to them, it's becoming a little bit of, you know, everything. And maybe that's, that's part of the problem. So, but I don't know with the community, I think that it is up to the community to start contributing to it. Like if I had time, I would, I definitely would. I would love to take on some of those issues and go contribute, but I get it. You don't always have time. I'm just gonna say something a little random

out of left field. I found that along this idea of how many different repos there are, from the main Kubeflow project, it's not being used anymore, and I just found that there are 42 other subsets that are being active and they are being used. So the whole idea of this question that came through

in the MLOps community chat or the Slack was looking at the greater GitHub repo and saying, well, it looks like the the activity has gone down here but really the other counter argument to that is yes it's gone down on the greater one but if you look at these smaller ones then you'll find where all the activity is actually buzzing still so then I'll ask you a direct question of sort of like a tool comparison. Even though Qflow has a lot of buzz in the last few months or so, when it comes to

at least personally in my mind, both the broadness of the solution set and popularity wise, Airflow has been much better, right? They are a much broader set of solutions and they are being much popular and much better open source and integrated with the Apache toolkit and the whole line yet. So obviously Airflow is there and then Luigi has been pretty popular too.

And then we got Ergo, we got MLflow. So there are so many tools. If it is going to be, most of them are so many tools. If it is going to be, most of them are open source. And if it's going to be open source, why that many tools and people are trying to diverge, you know, trying to create all of that. Isn't the time to merge at least few of them together to have the common functionalities. And then which one of those is going to win out? I mean, for now, I see that Airflow is a little bit popular than Kipflow in my mind. That's an interesting question.

Like, should, you know, should be moving towards, you know, creating sort of a unified open source project. I actually think, and maybe Demetrius, correct me if I'm wrong, but I know that there are some of our friends, I know Packaderm and some other companies are doing some work together to come up with what they would like kind of their view on a unified machine learning platform i'm not sure if it's enterprise only or if it's also open source but if it is open source that'd be really cool um but yeah i do think that there is you know definitely

some there's a there's a large ecosystem of tools it can be very confusing and and hard to make those decisions you have to evaluate them. I know for us at Benevolent, and I don't want to speak too much on their behalf, but I know that we really did our homework and looked at a lot of these options out there. And what fit our needs ended up being Qflow. You as a company have to go through that process to figure out what makes the most sense. You mentioned Airflow. I think it's great for certain tasks, basic data engineering, but for anything on the cloud, you have to do a lot of customization to make it work. It's not native to Kubernetes.

And that's actually one of the things I don't like about airflow. All of our workload, at least what I'm typically comfortable with, most of my workloads are running on a Kubernetes cluster. And I have to create a lot of things to make that work with Airflow. And it has a lot of problems because of that, because it's not native to that. Kubeflow, on the other hand, and let's say in particular, the orchestration tool, Kubeflow Pipelines is native to that. So it makes it a lot easier. And under the hood, it's actually using Argo. So Argo is a great tool to check out, by the way, because it allows you to build a pipeline that's native to Kubernetes. But it really is going to depend on... Right now, this is a little bit aside, but me and Demetrius are really trying to think about

what are ML Ops best practices? And the reason why it's related to tooling is because good tooling in many ways is the talking point for best practices. They encode best practices. And Qflow in some ways actually has a lot of good practices that I think encoded in it, but so do these other platforms. Like, you know, there's platforms that are devoted to just one part of that pipeline, like the deployment part.

And that's a very challenging area of its own, the feature engineering, so feature stores. That's a very challenging area of its own. And I think that it's great that we have focused efforts on that. But in my opinion, I do think that we need to start thinking about integration and the ability to build on top of these things. And yeah, I don't know how, Kubernetes is like the glue in my mind for a lot of these things. And so as long as it works with that, it makes it a lot easier to scale out and to build a reliable infrastructure.

Kubernetes is great for that. It's built for scale. And any tool that's not easy to work with like that is, in my mind, not going to last very long. And I don't know if David can talk much about this, but you can also use Airflow and cubeflow together and there have been people that yeah do that i'm using it like that actually so i'm using it like that uh airflow is an or as like an orchestrator on top of an orchestrator now some people may argue that's not the best way to use it that's a whole nother question but yes again but you have to do a lot of you know customization you have to build your own interfaces in between some of these things so it's it's yeah there's not a perfect compatibility but because of kubernetes there's a way to you know

use them in sync talking about this idea of all of these different tools it feels to me like what a lot of different vendors are doing and especially vendors who have their open source offering is they're trying to show the world or the cube flow community how they can integrate with cube flow and how they're starting to try and get involved in different cube flow projects because i think a lot of these different tools are starting to see that okay cube flow has a pretty wide range of adoption and it's being put out there's new versions of cube flow being put out i think that i had a guy on he was saying every 90 days so you're just getting a lot of advancement and so vendors are trying to figure out a way how can we get in involved in this because it feels like it is a rocket ship and it's going and it has it has legs I have one last question I was

talking to an enterprise engineer data data engineer, or infrastructure engineer. They were trying to productionize some of the models. One thing you mentioned, which was about Kubeflow, which was interesting, was we tried to implement multiple times. We failed multiple times, yet we can't give up. We somehow feel tethered to it. Is there any truth to it, or are you just making it up? Oh, yeah.

I think that's something that is very common and you have so many people, even on these threads where people are talking about Kubeflow, it's like, yeah, I tried that many times and I gave up. Or like you said, well, maybe they're so invested in it, they've already put so much time and effort into it that they can't give up now. And so like David was saying before, you have to have a team that knows what they're doing to stand this up. It's not the easiest. And I think that's one of the biggest downfalls that we hear about Kubeflow is to stand it up, the upkeep of it, just making sure that it works is a whole different beast than once you're inside and you're a data scientist or machine learning engineer that's creating models or serving with Kubeflow. Boy, that really, really sounds like Kubernetes itself, what you just described. I mean, you could just substitute the words.

And I think that that's, you know, so maybe that's what we need to look at. You know, the fact that, you know, Kubernetes also was a great solution, but it required so much time and so much effort and such an uphill battle to learn how to use it that the success of it hinged on the creation of sort of, you know, plug and play derivatives. You know, first you had, you know, things like Rancher, and then you ended up with, you know, VMware baking it right into, you know, vSphere with Tanzu. And now I think that it's all but assured success. Maybe that's what happens here. Yeah. And that's why I talked to David about this whole idea of, is it going to be the next, is it just one level of abstraction on top of Kubernetes and you have Kubeflow and then you

have the, it's the de facto, right? For the ML Ops space, it's Kubeflow. And then you have all of the different players in Kubeflow. I still think, David, I would love to hear you express what you told me the other day but it feels like we're a long way away from that yeah um i think so um i don't know this is it's very hard for me to like predict some of these things and maybe i just need more experience in the game more skin in the game to be honest and i i maybe let me also add this may this may be a bit snarky but i also think even for people who have been in involved in a space some time, I also don't think that they're capable of predicting the future that well. It's like, this is just a moving thing, you know, and it's changing all the

time. And some things are common. Like I've noticed that the, like this concept of pipelines is really important because it orchestrates a lot of components and that makes it a lot easier when you you experiment in an orchestrated way um but i don't know yeah in terms of like and rephrase it again this is about in particular about like them taking over or just just in general just adoption yeah more like if it was the one tool to rule them all and it was just what everyone used. Yeah, I don't think so. I it's, it's different. It's different than something like Kubernetes, you know, Kubernetes, it was

like, it's, it's like everywhere because of it solved a particular problem and it did it really well. So container orchestration and containers also solved a lot of those problems that helped out. I don't know if Qflow as a, an ecosystem of an ecosystem of tools will essentially take over everything, unless each one of those is just the best at what it does. And I don't think that's the case right now. I don't think that that's the case.

I think that there are some enterprise tools that are very good at what they do. And we need to recognize that. You know, it's not like we have to be afraid of saying that, you know, Kubeflow sucks at certain things, you know. It does because it's a work in progress. But that doesn't mean it's going anywhere. That's what I guess what I don't agree with is that just because it's not great and the best doesn't mean it's going to go away. And maybe that's just my over optimism.

But I just I yeah, I just I don't I don't see it taking over everything. I see it being I don't see it going anywhere. And I see more people continuing to work on the components of it that makes, make the most sense to them. One thing, last thing for me, I know you said that you see different kind of pieces where it's starting to congregate around some ideas like pipelines and feature stores and serving. Do you feel like those are going to be the main things? And do you have an idea of like those main things that are being congregated around?

I think the notion of, like I said, pipelines, so orchestration, I think the ability to string multiple components of a pipeline together is going to be really important. And any platform that aims to unify or like meet the needs of a stack the complete stack needs to be able to do that um i think i mean i feel like container is like the containerization is like the basis of like so much of this to make it possible because you decouple the runtime from the execution environment and that's really important um i i also see see what else the notion of a feature store being even more I

mean, it's already huge. And I definitely think it's super important. Like I feel like that solves a lot of common problems related to the data, which is actually one of the if not one of the biggest components of ML pipeline is the data that what you're doing with it. The deployment part is you can kind of treat it more like traditional software, except during the monitoring stage, maybe. And then maybe some other things around like how you keep it fresh and that model, you know, how do you continuously validate that, continuously train that model. But I think that the need, the ability to like wrap up everything from data extraction, processing, and then also down to deployment automating that we're gonna see more of that making it a little bit easier but yeah hopefully it gives you some sense of what I think but again

this is just my my limited my limited thoughts yeah so I think that's a great summary honestly David Demetrius do you want to summarize in a moment here is this the real deal where Where's this heading? Yeah, I've been thinking this for a while and it feels to me like Kubeflow is a rocket ship and I'm posing the question, is it going to be the de facto standard? So that's what I feel about it. I think it's a great tool.

It does have a lot of growing pains right now as we're looking at it and as we're seeing it grow. But once it comes into its own, and like I said, in two years, I can only imagine how robust and it's going to be much easier to use. It's only going to get easier right now. The early adopters are taking their chances on it and you're creating teams that are standing up Kubeflow. But I feel like it's not that far away from some kind of one-click deployment or an easy deployment that you can get. And so once that happens, you're going to start seeing a lot more adoption and it's going to be a lot easier. But what do I know?

Andy, any final words on Kubeflow? Yeah. I mean, it is, it has gone a long ways in the last at least a year in, in trying to help productionizing ML models. But the one-click deployments and some of the enterprise challenges, even security integrating with the main identity, authentication, authorization, who can see the models, who can deploy the models,

and how we can control the pipeline. It's not there yet maturity-wise for enterprise. Again, this goes back to my original point about, we tried multiple times, we failed. And that's the reason why they are failing because they are taking the model, which is built to make ML, productionize your ML models.

And then they try to fit into their enterprise model, which is where there's a lot of custom work. And if there's a way community can spend time in in doing that and making it easy for enterprises the adoption could be a lot more once big enterprises start jumping in yeah one final note i'll add to that is i also believe that the the success of this will be uh a function of the community and uh other companies enterprise uh contributing to this because it's like yeah that's how i feel like most of these big things work. You know, it's a collaborative effort. You know, it can't just be one sided. You need,

you know, being very practical. You need money. You need funding. You need like effort into some of these things. And that's like this is something even we do with the community level, right? Like we're a volunteer effort, making like even meet with something that me and Demetrius deal with. Yeah. So this is just like, you know, this is, I don't think it's that new. So I don't know. It's a little early in my mind to, to, you know, say anything definitive, but I'm optimistic. So thank you very much guys. Before we go, where can we find more of your writing and thinking on the topic of artificial

intelligence? Let's start with David this time. Yeah. So, so yeah, writing is kind of sp sparse, but you can be on the lookout for some more stuff. Me and Dimitrios primarily encode our content and our thinking in coffee sessions, so if you're really interested in hearing me and Dimitrios talk that talk, go to YouTube and check out coffee sessions for the MLOps community. Right now we're doing a series on a paper by Google called Continuous Delivery and Automation Pipelines. And we're really discussing a lot of these things.

So if you're interested, be on the lookout for that. Great, thanks. Demetrios? Yeah, one thing that I've started to do since the last time we chatted is take some of the great threads that have been put into Slack and immortalize them on to a blog or onto Medium. And so that's what I've been up to recently. You can check out the

MLOps community Medium. And that's probably where you're going to find the most. All right. And finally, Andy? Yeah, you could find me on LinkedIn or on Twitter at Andy Thurai or on my website, thefieldcto.com. Great. And you can find me on Twitter at sfosket and you can find my writings at gestaltit.com. Thanks for listening to the Utilizing AI podcast. If you enjoyed this discussion, please remember to rate, subscribe, and review the show in iTunes since that helps our visibility. And please do share this show with your friends.

This podcast is brought to you by gestaltit.com, your home for IT coverage across the enterprise. For show notes and more episodes, go to utilizing-ai.com or find us on Twitter at utilizing underscore AI. Thanks, and we'll see you next time.