Utilizing Tech - Season 7: AI Data Infrastructure Presented by Solidigm - 05: Building AI and Machine Learning Infrastructure with @GMinks

Episode Date: September 22, 2020

Stephen Foskett is joined by Gina Rosenthal, an expert on enterprise IT infrastructure and operations. Gina has made her career in enterprise IT infrastructure and has worked with many of the largest vendors. In this episode, she considers how vendors approach artificial intelligence, what applications they are delivering, and what this means in the enterprise. The conversation turns to ethics and risks of AI applications and how business should approach building AI models. As AI applications are deployed in the line of business, IT infrastructure organizations need to be prepared to handle the demands of these systems with next-generation cloud platforms. This episode features: Stephen Foskett, publisher of Gestalt IT and organizer of Tech Field Day. Find Stephen's writing at GestaltIT.com and on Twitter at @SFoskett Gina Rosenthal, Founder of Digital Sunshine Solutions. Find Gina on Twitter at @GMinks Date: 09/22/2020 Tags: @SFoskett, @GMinks

Welcome to Utilizing AI, the podcast about enterprise applications for machine learning, deep learning, and other artificial intelligence topics. Each episode brings experts in enterprise infrastructure together to discuss AI in today's data center. And that's really the focus of today's discussion. I'm very pleased to have a guest who knows an awful lot about enterprise architecture and next generation architecture, and also the vendors who are delivering enterprise technology. So let's meet her. Gina, why don't you introduce yourself? Hey, hey y'all. I'm Gina Rosenthal, and I am the founder of Digital Sunshine Solutions, and I help companies look at how they're talking about these types of things, how they're talking about what they're bringing to market and how they can help other companies with things like AI.

You can find me at GMinks on Twitter and DigitalSunshineSolutions.com. Thanks, Gina. And I'm Stephen Foskett, organizer of Tech Field Day and publisher of Gestalt IT. And you can find me on Twitter at S Foskett. So Gina, your background is really interesting because you have not only worked in the end user space in terms of, you know, enterprise IT, but also for a lot of the vendors, including some very big and very important vendors in this space, some of the largest vendors of enterprise tech. And so you've watched this whole evolution of enterprise architecture from the data center to the hybrid cloud, you've seen technologies

coming into the data center, and it seems to me that you're avidly watching as new technologies emerge and helping companies figure out how they should use them. So when I talk to you about AI, I guess let's start with what does that mean to you from an enterprise IT perspective? From an enterprise IT perspective, I think that's something that's really emerging, AI. And I think when you talk about AI, people are saying AI and meaning so many different things but in general when I think about it AI has underpinning architectures of HPC machine learning and deep learning and in very simple probably too

high level speak AI is teaching a machine to do something that a human would normally do to make decisions that a human would normally do, to make decisions that a human would normally make. So there's lots of opportunities for enterprises to think of how that type of technology could help their customers or help them do better at whatever it is that they do. So, yeah, and I think that that's sort of the, I guess, the classic definition is how can computers and computer systems in a way replace human decisions, not necessarily just augment human power. And, you know, I think that it's good to have a broad perspective on it because you know in many cases when vendors start talking about AI they mean one specific thing right you know they a lot of the time well a lot of the time it's AI washing but if it's not you know

what kind of AI applications are you seeing now in the real market for the enterprise for the enterprise I think you have to be careful because there is a lot of AI washing, but there are applications people are creating. I think one of probably the most prominent ones is to catch spam, not spam so much, but to catch threats to their networks or to their environments and being able to

learn. So think kind of the Bayesian filters we had 20 years ago, but actually teaching a machine to teach itself over and over again, training it that this is bad. Let us know if this happens, let us know this happens, come and get us. So a human can intervene. But if this range of stuff happens, you should know what to do with it so I think that's probably one of the most common ones that normal enterprises are able to use right now there's all sorts of other things though I mean there's if you think about autonomous cars that's not typical enterprising but it's definitely something that is a, I bet if you

think about if I'm the enterprise architect for a big city and a transportation system, maybe that is something I need to look at. Should my buses be able to drive on their own? What's the impact to my citizens if I allow this to happen? And so I think there's all sorts of ways to bring it in. One area that I've seen a lot of people talking about is in ERP systems, basically bringing AI in to help make decisions, sort of, you know, workflow decisions or production, supply and demand decisions. Do you think that that's going to be a major focus for this technology? Maybe. And I think, you know, that can, some of those kind of, I think, also run the line of not really being AI, of being more, just having more powerful machinery that's able to crunch more and more numbers and do pattern matching more.

Maybe. And I think another one, you know, insurance is another big case where it can be. I think all of that remains to be seen because some of the places where it seems like it should be pretty easy to remove the humans from making the decisions, it turns out that if humans aren't making the decisions, there are lots of unintended consequences still. So, because we don't program anything without, everything that we do in technology has our own biases, even if the biases are ignorance, you know, built into it. So there's, that remains to be seen to see like how people are impacted personally, if it's good or if it's bad, to see if that keeps going forward.

Yeah, that's actually one area that concerns me overall. And certainly in the case of like ERP systems, you know, if you're only driving using the rearview mirror, and in many ways, like machine learning really is only driving using the rearview mirror, you know, you're only, it's only driving based on what it's seen before, what happens when it encounters something that it has never seen. And, and frankly, you know, I think that a lot of the AI experts might say, well, you know what, it's actually going to be surprisingly effective at handling those situations. And maybe it will be, but maybe it won't be. And it's awful hard to know, because basically, how would you know? Nobody knows exactly what the system is going to do.

And that's actually a huge point of this, right? So the way that the systems learn is, it's very interesting, just even the terminology. So this whole field is not brand new. It's got 30 years behind us. You've got big universities backed by lots of government funding, running enormous bare metal clusters, running HPC architectures, and running machine learning, deep learning on top of that. So the terminology is very interesting. So the data scientists look at the problem and say, okay, I want to build

an application to solve this problem. Then they decide which type of neural network they're going to use. And they named it that because that's what they're trying to do is to teach this machine to think like a person. And the neural networks are a bunch of math, being very general, right? But like lots of math, they put weighted decisions here and there. They find a great big bunch of data. They run it through this neural network. They run it over and over and over again until they get the percentages where they find

them to be appropriate as far as the neural network spitting out correct answers. Once they do that, they give it a bunch of untrained data. They check it again and see what that percentage is of correct or incorrect data. And I think that that is what the data scientists do. The data scientists pick those neural networks. And that piece of it's very important. It's probably very particular to the risks involved to humans or to the business about what that percentage of right or wrong as they're doing the training can be. So if you're in a car, you want that to be like a very almost zero.

You don't want anybody to die, because you didn't do enough training on it. So I think that's, it all just depends on like, what is the business outcome from this application? This is still IT, it's still our, the things that we forget to think about when we're talking about, I'm building this architecture, okay, fine, but what's the architecture for? And what is the result? It's the same thing with AI, right? It's what do I want this application to bring to me for a business result?

And what are the risks I'm able to take? And when they design the neural networks and when they train the machine with trained data and then they train it with untrained data, they're looking for that percentage of how right am I and how few times am I wrong. Yeah. And I think that it is interesting. I think that to use the autonomous driving as sort of a metaphor, because that's in a way what we're talking about doing. It's just not driving a car. So, you know, you know, autonomously driving a car on the road is something that I think a lot of people can understand and you can comprehend that the machine. Is doing that and you you know you've taught it how to recognize you know images coming in from a camera.

In a way that allows it to react appropriately, even though of course it's not conceptualizing what it's seeing the way we would. It doesn't know that that's a bicycle. It knows that when it sees this pattern, it should avoid, you know, driving into that thing. And in a way, I mean, autonomous driving is really what's happening, whether you've got an ERP system or a firewall. You know, it's autonomous driving. It's looking for patterns. It's reacting to patterns in a way that allows us to get a desired outcome. And I think that that's really, you know, kind of if we extend that metaphor outward, one of the things you said, I think is important as well.

And that's that, you know, you need to have some guardrails around it. You know, maybe an autonomous driving system can only work on the highway and controlled access sort of way where you're not going to have somebody driving like across the street in a bicycle or something, you know, you wouldn't encounter that on the highway. So maybe the machine doesn't need to know that. Similarly, maybe a firewall system doesn't need to know how to handle every possible type of traffic. Maybe it only needs to understand web server traffic and you can feed it, you know, web server logs from years and years of access and tell it, you know, find anything that's unusual here and not say, oh, and now we're going to put it on our VDI application or our banking systems, you know, because of course it wouldn't have no idea how to

deal with those systems, right? Right. And I think that's the beauty of it. So like, you know, I think that having this, you know, backing up and having this understanding, this is a new type of applications being designed in a way where, you know, we as typical enterprise tech people aren't used to architecting for. But these applications can do some pretty cool things. I mean, this is how they have been able to, you know, the scientists working collaboratively together have been able to identify and zero in on COVID-19, what it actually does and how the actual RNA works. And they're building models to, you know, that's what they're going to put in the vaccines. on this, it would have taken them so long to do this if they didn't have already these models built and have all the data accessible. And I liked what you said a minute ago about this historical data. Like that's amazing. You've got some, you've got some, you know, verticals

that have data that they've digitized that goes back hundreds of years. Think about just retail. If we ever get to go back to stores, you know, this is one of the things the retail folks have been working on is, okay, if you walk in the store and you're a customer and you have our app, we know that you're here and we can tailor your experience through your phone, which becomes an IoT device,

to all of our devices that we have throughout the store and we can take all of the experiences of our customers every night pull that back to our historical data over the years of all the times you've shopped with us we can crunch and crunch and crunch and figure out how to make those experiences even better for you which impacts our supply chain our prices to you how we staff the stores what services we offer all of those things so there's you know it's not all gloom and doom there's really cool stuff that can happen however like even just that situation i just described that's not a typical enterprise tech thing we would think about architecting right now and that's the thing is we've got to

break out of this idea that it's just another application. It's just a cool thing to learn and think about what could happen. You know, if you expand your mind of thinking of how a network can work and thinking of what is a device and how could those can merge together to, to take data, continually feeding it through to, to produce something cool. Yeah. Well, and that actually leads me to another question for you. And, you know, based on your experience in enterprise tech, you know, most of you and me, you know, most of the things that we've worked on are more on the infrastructure side,

the IT data center side of things. And of course, IT infrastructure serves the line of business. So in your opinion, do you think that AI is going to have a bigger splash early on in the infrastructure side of things or on the line of business side of things? So like an application like you're mentioning, like retail or something like that versus a firewall or something? So what I would hope, right? What I would hope is that everything we've learned from the DevOps wars and all the rest of it is that this would change and that ops and the line of business are working closer together than ever before.

So if someone comes up with, I'm the app owner and I'm the orchestrator for all of these things, maybe that app owner has IT people that are his partners, her partners, and they work together super tight. So when they come out and say, okay, here's the idea I have and it's going to be in the stores and we're going to take this old data and we're going to do this and we're going to spread it around. Then the IT person thinks with their, you know, their ops hat on, okay, well, you're going to need a firewall and you're going to need this and you're going to need all the rest of it. And how can I make all of this work for you? Or if the solution is we need to simplify things in the data center, so help us, we're going

to figure out how to make this firewall work i think it's it i think what's going to happen early on is we'll get a whole bunch of ai watching and we'll see all sorts of new devices in enterprise software that has ai built into it so i think people need to understand what it is so that they can do their own evaluation if that's if they're getting the best bang for the buck and if there's really AI or if they're spending too much money for a code word but I think over the next three to five years that we have to start working with app owners closely to protect them from not having the infrastructures they need to support in a very resilient manner what they're wanting to build. So do you think that, you know, applications of AI technology, when they do come to the line of business, do you think that that's going to run on the same infrastructure that

we currently run, you know, web servers and application servers and things like that on? Or do you think that it's going to require a special kind of infrastructure? Well, yes and no. I think both, right? I don't think it can be the same. We're not going to build on three-tier infrastructure, right? We're not going to build on server switch storage. And we've seen this kind of moving already. And we see in the cloud, we see already the infrastructure moving to a more cloud-like environment. And we see all sorts of startups now that have these very super scalable file systems, and that's what it's for. So you need a super scalable, a way to have a super scalable file system, whether that's on storage or on all servers or however it's going to be,

and then you also have to have a very fast network and you have to have very fast communication between whatever the nodes are. So if this is built on all just bare metal, if it's built on VMs, if it's built with containers, which is most likely the case, all of those nodes, depending on the type of neural network that you've built, have to talk to each other back and forth to do all that crazy math and the weights and everything to do the deep learning than the machine learning. So you have to have very, very, very fast machines, which is why you want GPUs, FPGAs, like those kind of more modern components. Okay. Yeah. And I think that from an infrastructure perspective, you know, I'm looking at it and thinking, you know, GPUs, GPU sharing, that's obviously, you know,

composable infrastructure. That's obviously a specific technology that can be leveraged in AI workflows. And as well, you mentioned storage, you know, you need very high performance storage, very scalable storage. That's a technology that can be leveraged in AI workloads. You know, are those, is that the right way of thinking about infrastructure for AI, that it's going to be more like composable infrastructure

and scalable storage? I think so. But I also think, you know, think about like we were talking about having this old historical data that you want to bring in. What's, what's, you're going to have to have this data that's going to live someplace. Hopefully you won't have it all on tapes. Maybe you can pull that into someplace. Where do you pull that into? And how do you make that available? And how do you make it secure? so if I have a data set that I want to give to my interns and it's the same data set that we're using for

production like how do you how do I get these humongous data sets to the different teams that need to use them to run through as training models so yeah so you've got lots of things to think about. You've got the application that gets presented to the end users. You've got the machines that are actually doing the crunching of the information. You've got potentially older information. You've got potentially, you know, these IoT devices, which I think is all part of this discussion, right? Getting new information off all the time. Tanji, you've got people shedding information that you want to grab that information and make that part of this whole system. And well, it's not even information, it's data. You wanna grab that data

and make it part of this whole system that you can up-level and turn into information and leverage that for whatever business purpose there is. So I think it requires us, it's really important to understand how we've been designing in the past and where all those data sets are, but also important to understand the new structures

that we're seeing emerging and understanding how do we blend them together and in a very agile way so that we can make sense of all this data that we have. Yeah. And I do think, you know, kind of sort of closing the loop here on enterprise infrastructure, it seems to me that there's going to be an explosion of, I guess, data reuse or not, you know, data recycling. I don't know what you want to call it. But basically, enterprises are going to see that they have data that they aren't leveraging right now. And then they're going to try to figure

out ways of exploiting that. It reminds me just personally of a story when I was, I used to work for a major fuel retailer in Texas. And basically at the gas stations, they sold all sorts of snacks and drinks and so on. Well, the application that I was supporting was the first time that such a retailer had ever bothered to track basically what was selling where and when and by whom. And so previously, I mean, they had records that we sold a snicker bar and a bottle of Coke, but they never bothered to use that information in any way. It was just inventory. Oh, well, we need another one.

And then they said, well, what if we build a data warehouse? What if we then query that data warehouse? And what if we then try to find correlations in that data and figure out maybe Texas, they like Dr. Pepper, but maybe in Ohio, they like Mountain Dew. And so we're going to ship more of that to these different places. And it was a revelation for the business. And basically their retail sales took off. I could absolutely see that happening with AI where basically they sick an AI on this system and say, okay, here's everything that's been sold. You know, when, what was the temperature, how many people were in the store, you know,

all sorts of other data points that we may not have ever looked at. How do we make this interesting? How do we make some useful judgment on that? Is that the direction you think this technology is going? Absolutely. And I think the reason why it hasn't been there is we have, what's happened is, think about what's happened with just server and storage components over the last seven years five to seven years they have dramatically increased performance and there are things that are just

crazy the stuff that it can do um so we haven't had the the hardware that's enabled us to take all that data and do something with it i'll give you an example that I heard from a customer, another fuel company, right? They had stores. I want to say it was North Georgia, but it was out. Maybe I just had that in my mind, but it was someplace out in the sticks. They would send that they send the fuel trucks out and they would go on the route that like, okay, today's the day I go to this County and, you know, go all the rural routes over here and do what I have to do. But they were able to start, they had some sensors on the gas tanks, you know, the big tanks underneath, and they were able to tell who was running out

of fuel first. So instead of going in this route where it was like this circular, which would make sense from a, I'm driving down the road kind of view, they would go first to the people that needed it. And so it was more logical and people got, nobody ran out of gas, nobody got close to it. Everybody had what they needed when they needed it because they were able to use an AI application. So I absolutely think that's what's going to happen. And it's a little scary to me because there's so much information that we shed. Do we really want people to have the ability to connect all those dots? And whole other ethical side of that. Like, that's a little scary. But yeah, it's, it's possible. I absolutely think that's what's going to happen.

Yeah, I think it's, I think it is interesting to think about that. And also to basically apply the lessons that people like yourself and I have learned over the years and watching this technology. Not to sound too Bertrand Russell about the whole thing, but I would say that the technology tends to find its own way and to find its own path. And basically the availability, I think if there's a lesson from enterprise tech is that the availability of systems, you know, processing, storage networks has basically found a way for us to leverage that in different ways. And I think that that's what we're going to see as well with AI in the future.

So it will be interesting, I think, to circle back with you, you know, in a year and say, like, how has this changed since the last time we spoke? But I really appreciate you joining us today. Where can people connect with you and follow your thoughts on enterprise AI and other topics? My website's really great, digitalsunshinesolutions.com. And I'm on Twitter, G-M-I-N-K-S. And I interact with everybody there. I'd love to talk to people about this. Thank you for listening to the Utilizing AI podcast. If you enjoyed this discussion, remember to subscribe, rate, and review the show on iTunes,

since that really does help our visibility. And please share this show with your friends. This podcast was brought to you today by GestaltIT.com, your home for IT coverage across the enterprise. For show notes and more episodes, go to utilizing-ai.com or find us on Twitter at utilizing underscore AI. Thanks, and we'll see you next time. you you you