Orchestrate all the Things - H2O.ai introduces Hydrogen Torch to bring the grandmasters of AI for image, video and natural language processing to the enterprise. Featuring CEO / Founder Sri Ambati

Episode Date: February 21, 2022

H2O is an open-source based AI powerhouse with a grand vision. The latest addition to its product line wants to bring the AI capabilities of the web giants to the rest of the world, says CEO and Founder Sri Ambati. Article published on ZDNet

Welcome to the Orchestrate All the Things podcast. I'm George Amadiotis and we'll be connecting the dots together. H2O is an open-source-based AI powerhouse with a grand vision. The latest addition to its product line wants to bring the AI capabilities of the web giants to the rest of the world, says CEO and founder Sri Ambati. I hope you will enjoy the podcast. If you like my work, you can follow Link Data Orchestration on Twitter, LinkedIn and body. I hope you will enjoy the podcast. If you like my work, you can follow Linked Data Orchestration on Twitter, LinkedIn, and Facebook. Thanks, George, for having us. We basically

started as a grassroots movement, building a lot of community around data science. Our core mission was to make data scientists heroes of organizations and be part of how organizations can be transformed. So our mission to democratize AI, H2O, and it started in, I've been working with AI from my early days in doing voice to text translation for Indian space research a few decades ago. And then I came first in touch, contact with neural networks at the time and the instabilities that they had. And I think over time, over the years, it's super exciting to see the changes that the maturity in the AI, both modeling standpoint,

as well as accuracy and speed and ease of use for ordinary citizens. So, and I'm from the Silicon Valley, born and raised in India, but like every immigrant, spent a lot of time in startups here. And between startups, I'll be spending time in sabbatical between Berkeley and Stanford. And through that, came in close contact

with some incredible mathematicians and physicists and great compiler and systems engineers. And bringing all of that together, we melded the software of H2O. But it was not until my mom got breast cancer that I was really inspired to go democratize machine learning for everyone and make algorithms easily accessible

at the fingertips of every physician or data scientist, solving problems of real value for the society. So the amount of data being used for lumpectomy versus mastectomy was very small. And so we needed to build, reinvent math and analytics at scale. And that led to H2O, the grassroots movement

that brings compiler engineers, systems engineers, physicists, mathematicians, data scientists, grandmasters together to make it easy to build models of high value and high accuracy very fast. And that ability to democratize AI will be faster, cheaper, easier and make our customers AI superpowers or AI companies

and use their data to build really rich brand and communities. Today, we are fortunate, 20,000 companies use H2O and more than a million data scientists. It's easier to get a data science job or make a great living with data science using H2O as a skillset globally.

And super, super impressed of the support of the customers and the investors and the community. And we're extremely grateful for the co-creation opportunities that we have with our customers between the likes of AT&T to Cornell Bank of Australia to Unilever, Reckitt and Capital One, PayPal, Intuit and others. So we're super, it's just still early days for AI and we're super fortunate to be here to be able to

be that shepherds or steward of trust, building trust in AI and dehyping the space so customers can build rich AI-first applications and continue to build great, strong customer communities. Great. Thank you for the introduction. And I guess H2O is one of those companies that have been growing alongside AI itself. And so since the occasion today is the fact that you're about to announce an addition to your product portfolio, I thought the next logical step to take before going to the specifics of the new release would

be to have like a quick walkthrough of your existing product portfolio and then we'll try to see where exactly the new product fits in there. So you have a lot going on obviously, you have the open source offering, you have the AI cloud, you have what you call driverless AI and your AutoML offering, you have Document AI, you have Wave, you have Sparkling Water, Feature Store and MLOps. And I think I got everything. If I missed everything, feel free to jump in and correct me. So yeah, if you can just give a quick overview of your offering. Yeah, if you think about machine learning as the assembly language, the core math. So when we started, as I mentioned,

we were one of the first.ai domains. There was not much machine learning in open source that was very scalable. There was languages like R and Python that allowed customers or communities to build models, but they were very slow or brittle or not fully featured. So we built the world's fastest, I would say, distance calculator. And when you can calculate distance between two long tensors, you can now

start producing rich linear and nonlinear math across different dimensions, high dimensional and low dimensional data. And those are broadly scoped out in the H2O framework as our open source contributions to math and machine learning. And that's the lowest layer, we call it, I would call it the assembly language for AI. Then we realized we were super fortunate that we could actually bring others

to join our open source movements, like XGBoost or TensorFlow, ByTorch. And one thing led to Scikit-learn. So one thing led to, and we started contributing to some of these projects ourselves. And one thing led to the other, we realizedarn so one thing and we started contributing to some of these projects ourselves and one thing led to the other we realized that now the sum of the part we can build a whole system that can take advantage of the best degree in this we put on the broad category of innovation called auto ml where we are roughly using the best in class brainpower of solving time series and IID data, which is a typical enterprise companies have problems that they want to solve around, say, predicting churn or predicting

fraud prevention or subprime credit scoring or figuring out ICO transfers or not. These are all problems which are transactional time series data. So we built automatic machine learning for that space. And that layer output that as compilers of AI, where they're tuning the parameters, finding the right features automatically, preventing the common pitfalls of making AI like overfitting or making sure that we have the right cuts in the data, right?

And really running a genetic search over the space to find the best model, the best feature and the right ensemble, and then generating the code that takes it into production for inference called Mojo Java code. And that is the automatic machine learning. We have two products there, one open source H2O AutoML and that uses H2O and XGBoost.

And other that actually is a much broader offering called driverless AI and that is closed source. And driverless AI has been the engine of our economy over the last three, four years. We've acquired hundreds of customers and solved thousands and thousands of problems where the out-of-the-box accuracy of the model. And we divide the problem space into a lot of recipes. And each recipe from different, several of our Caggle grandmasters who are at H2O. And the grandmaster, think of them as,

they're in chess, there are 1200 grandmasters in AI and data science, there are about 250. And H2O is incredibly fortunate to have 10% of the world's grandmasters as employees and as makers building new products. And in this case, they build recipes to make machine learning more accessible to new data scientists. I wouldn't put them novice and new data scientists who are aspiring to build models of high quality

that would otherwise be difficult to build for unless you have decades of experience doing it. The next phase of all of this is now that you generated a very rich inference engine, you managed to build a good model, you want to then operate this model in a safe fashion because data has data has inherently has bias and decisioning over the last hundred years has bias captured in those stations so trying to prevent a biased model going into production or prevent a model that has blind spots adversarial testing validation how do you build a very safe environment

for deploying model and then integrating it to the CICD of software building. That phase is what we call the middleware for AI. And we roll that out as H2O AI hybrid cloud for on-prem AI cloud on cloud and for the edge and for serving on the edge. So that kind of is the middleware for AI.

And that's where you're seeing H2O AI Cloud. And the way customers use AI in the cloud is through applications. And that's the way the AI App Store and Model Store, Prebuilt Model Store and Feature Stores, which are the insights coming out of model building. That's kind of where you're seeing the pre-built feature store,

model store, and app store. And I think our customers needed a low-code environment to produce those applications relatively smoothly. So we released open-source H2O Wave, which allows customers to rapidly code. It's like an SDK for building applications. And that's where you're seeing the H2O Wave, another open source framework. Along the way, wherever we could, we tried to continue to strengthen the freedom for the end users by giving open source. For example, Data

Table was a powerful package for data munching in our community. We brought that to the Python community so customers can build that. Some of our machine learning methods were very strong on CPUs. They couldn't take advantage of GPUs. So we morphed RH2O for GPUs, which again, we chose open source for that and collaborating with the likes of NVIDIA and others to make the

GPU version of software even stronger. But I think the net net, wherever there is math, wherever there is SDKs and ability for customers to essentially rebuild, they need the freedom to innovate. We chose open source. And when the customers are happy to kind of allow us to co-create with data, we have done co-creation with data as well. But that brings us to hydrogen torch and why hydrogen torch. So as you can imagine, George, data is at the heart of AI. Data fuels AI.

And ability for domain and data scientists to co-create together is actually captured, very well captured in some of the recipes. But historically, our work was focused on where the customer's current pain points were, which was with data that is hundreds of dimensions, thousands of columns, or even 10,000 columns. Like in Macy's, you'd have 10,000 products being repriced.

In a time series data, you're probably looking at thousands of sensors. But once you get to the life sciences data, you have 100,000 to a billion columns. So at each boundary, we would reinvent the entire platform to make the platform really fast. Speed and accuracy are super important for us.

And that's why memory work was super necessary to make things fall, hopefully sometimes in L1 and L2 cache lines so you can get really nanosecond speeds and sometimes in domain memory. But what one class of problems we were always passionate about personally being NLP, coming from NLP

and coming from image recognition and video and audio is that that class of problems is what hydrogen torch addresses. And that's kind of the story of Hydrogen TORCH, where H2O now walks into the traditional space of the web giants like Google, Microsoft, and Amazon, and Facebook, and roughly uses some of their innovation, but challenges them with an innovation that allows customers to use deep learning more easily, both taking pre-built models and transforming them for their local corpuses.

Okay, great. Thank you for the extended overview of your products that you provided, which by necessity, I guess, had to be extended because you do have an extended production. And thanks also for the introduction to Torch. And we're going to return to that, obviously. But before we do, I wanted to take a step back and ask you a few clarifying questions on some of the things that I see as the key premises behind what you do. I think one of those which you

briefly touched upon yourself is the in-memory engine which is something that lends speed and performance to what you do. And I was wondering if you'd like to share a few words on its background, basically, how it came to, how did you conceive it, and how it came to be implemented, and the ways that it's used to power your platform today. Thanks, George, for the double click on in-memory. as we can imagine that data, using a lot of the data and typically in some of the use cases, it's a needle in a haystack, as they say, that one in very imbalanced data sets, is one in a million or one in 10,000, 200,000 occurrences

that you wanna detect, whether they're anomalies or model around fraud. So when you're dealing with that size, rare occurrences, we call them in balance data sets, you probably want to put as much data for training as possible. And I think that's where in-memory really lends itself to doing these models rapidly and fast. So what we ended up doing was, my background comes system without putting a lock is going to be super important. So we created almost a lock-free and mostly rate-free distributed synchronized system that can count efficiently at scale. So I think that was a key innovation in making some of the techniques like

boosted machines, scale. Boosting machines are this idea of using a very weak learning system that can continue with a good exposure to data. It gets really very strong. And in this case, gradient boosting trees, they would produce thousands of trees on data. And with enough trees, you probably capture a lot of the data. Of course, you want to prevent a bit of overfitting.

But that scalable machine learning system is backed by a continuously, the splits are calculated at scale on large number of points. And that innovation is possible because of the distributed shared memory system that we created behind the scenes. And it's a chunked dataset. So you're creating chunks. And I remember the first time we walked into Macy's

where that chunking needed to be variable-sized chunk because otherwise if you had very high-dimensional data, you would consume a lot of data, a lot of memory. So the columnar compressed chunked in-memory representation of large data was super important, which led to kind of the high scale or ability to process billions of rows

in a very short span of time. So machine learning became more automatable. If you think about why we started this in the language of AI with ML, making it very fast. Once we made these GBMs fast and then generated the scoring engines for them, we call the Java code or Mojo or C++ code for them, or even sometimes ARM edge processor code. Once we generated, auto-generated this low latency code

that can fit in L1, L2 cache lines for decisions, we now made this transition from science that is practical, that can be run at scale on real-world data sets, and then transforming it from that to software which can run on the edge or inside the application. Because if you think about the whole thesis here, the convergence of data analytics and apps, the DNA of any enterprise is super important because decisions and data are happening at the, data is being made at the application level.

And for true AI companies, they are making new data all the time and taking advantage of that feedback loop to build self-driven or data-driven organizations that are self-learning. And that organizations are able to create absolutely new data. And so the in-memoryness makes it real-time. So people can absolutely learn on the edge and also train on the edge and score on the edge. And this leads to another innovation that is in the hopper around federated AI

and being able to learn scalably across multiple machines and multiple servers. But the core innovation of H2O, the open source platform, was to actually make kind of math scalable, right, by putting it in translating traditionally on-disk slash one-core, in-core calculations to multi-core,-system and scalable memory based quantitative learning. Okay well thank you and actually the last part of what you mentioned

about integrating while offering this innovation as part of the open source core of H2O precisely touches upon what I wanted to ask you next. So I understand some parts of your offering are open source and some parts are proprietary. And at the same time, you're also obviously operating what you call an AI cloud. So I wanted to ask you if you have any insights as to what segments of your users do you see utilizing the open source part versus the proprietary parts, and whether, besides your own hosted cloud, whether you see demand for self-hosted services, whether dictated by regulatory compliance

or security issues or anything else, and how you are addressing those. Yeah, I think the way we think of open source, open source is about freedom and not just free, right? Sort of some, and as you can imagine, mathematicians and physicists are kind of like not too different from artists. They prefer to see fully what their model is doing.

They wanna know that a random forest is a random forest. Stochastic gradient descent is a stochastic, SGD is an SGD, right? So they wanna know why the quantile, how it lead to the cuts. And so math, and also we think of math as unpretendable fundamentally, and also something that was given to us over the centuries

right sort of how starting with the beakers to kind of from one gym to to the latest and greatest innovation from these last few centuries and so we think that math needs to be almost, if we unlock math and make it accessible globally, that will unlock discovery at the highest level. And we as humans will be able to innovate faster. So just like the beginning of the century, last century, we had quantum mechanics unlock all the innovations that came that century, we think that the ongoing revolution in democratizing machine learning methods will unlock discovery for the next century. So that's why we chose to keep that in open source.

It was a philosophy. And historically, analytics and data and apps were completely divorced from each other as three different stacks in computer science and the industry, and we chose to bring them together. So that was more of a deliberate kind of choice on our side to democratize AI. And today we have a huge AI kind of revolution

going on in every walk of life because of that choice that we fundamentally made in 2011, 2012 to bring AI to everyone. And open source was the fastest way to do that. We have millions of data scientists using our product every day. 20,000 organizations advertise H2O as a necessary skill set, including our own peers, competitors of the current time to the competitors of the future, and past used H2O as a core machine learning method. We were surprised the other day when AWS analytics team advertised for a job description with H2O as an S3 skill set alongside TensorFlow and Torch. So that's the open source and we have competition. It's healthy competition there.

Unlike other open source monopoly thinking, we actually create open source to create, do good and be there where we are needed. At the different layers, now, driverless AI is a closed source product. Our monetization as a company comes from driverless AI. It's a layer above. It's a completely new 35,000, 40,000 commits new product line. It also complements our open source by doing Python first, C++, CUDA, our open source, H2O is made in Java, but it's fast Java, but it's in Java, so it's platform independent. But I think we chose to take advantage of platforms like GPUs and TPUs in our closed source innovation and driverless AI. So there we flipped to CC++, Pythonic first, because Python has taken off so vibrantly in machine learning.

And then if you think about the H2O AI cloud, we bring all of the open source, closed source together, but allowing customers to build applications using Wave, which is open source, because SDKs, we think that they need to give the freedom to end customer. So low-code environment that they can use Python to build rich visualizations for their core applications. And I think that's a place where we essentially let and bring open source back. Data munching, another very, very difficult problem. 80% of all AI or machine learning users needs data munching their data table. We chose open source. So we kind of made sure that the customers have the best,

most freedom and best of breed. And when you think of the app store, you now have a hosted environment, managed cloud, hybrid cloud on-prem, multi-cloud, so it can run anywhere and allows customers to get the best of breed kind of microservices that they can serve as with model ops and deliver, use the best AI engines. And Torch follows into that playbook where it will be one of the few AI engines that we are building and hydrogen torch. I think overall, that's kind of where the open, closed and cloud. So for any serious company of any scale today

that is very serious in software, open source is not just an option. It's a must-have to build great communities and allow for the freedom for customers to use AI. And I think it would be almost a necessary step in AI, in machine learning to have open source. Okay. Thank you. have open source and cloud. Okay, thank you and I guess to simplify what you elaborated on,

you are also able to serve customers that want to have a self-hosted environment and so that covers a wide user base. But I guess it's about time to actually focus on what you are releasing now. So on Torch, I guess, well, part of what you covered already has to do with what it entails. So again, to oversimplify, models to cover, to deal with image and video processing and natural language processing. And you also mentioned that a good part or perhaps the entirety, I'm not sure you can clarify that, of those algorithms has come to you through Kaggle and through some of your employees that actually work with Kaggle. And so I would like to ask you to elaborate a bit on that process and also as a follow-up the kinds of, if you can get a little bit more specific about the types of use cases and algorithms that Torch includes and supports. Thank you. Seems like George you're able to bring most of the story together which is quite

fabulous to see. The way to think about the, I mean, just to clarify on the on-prem stuff, a large number of our customers in regulated industries are on-prem still. They are going to the cloud and we are very excited for that because that will unlock more innovation. And we are meeting them where they are,

right in the cloud, so that they can take advantage of a multi-cloud. They don't have to pick. Many of my retail customers don't want to go on to say Amazon. And if you're like Hy-Vee or a H-E-B grocery chain, you're competing against the traditional Whole Foods-like ecosystems. And if you're in the ad tech space, you're probably just similarly worried about Google. So I think there's a lot of times when customers want multi-cloud and that's what H2OI cloud offers. And the best AI, best auto ML that they can get, best vision, speech and NLP with hydrogen torch on any cloud. And so that allows them to kind of make sure

they have a data cloud and an AI cloud that allows them to future-proof their business in the event that tech giants come into their space. So that's kind of the core ethos behind the cloud and open source. Double-clicking into the Torch site doesn't make sense, George, on the cloud and the on-prem side. So I'm going to the Torch side. George Lambertis- Yes, yes, yes, absolutely. Thank you.

Sanyam Bhutaniyaraman On the Torch side, I think there's some very interesting... I'll start with a problem that was brought to us by the Singapore government. And here's what their phone was. They were actually trying to install ways to prevent an accident from before it happens or look for rebuilding as we are rebuilding from post-COVID. They're trying to understand how can we kind of see if traffic has picked up or how can we see if someone is driving in a way that can cause them get them into an accident. So that's those are the problems that triggered this particular problem where we are looking at taking traditional big models and pre-built models and

then fine-tuning it the corpus of camera video they have in this case. Of course it's Singapore and they were really trying to make sure that they're kind of helping their communities on the island Sentosa. We know if you think about the traditional use cases for torch, a lot of our customers have access If you think about the traditional use cases for Torch, a lot of our customers have access to,

or Hydrogen have access to Torch itself, which is by Torch and from Facebook and Google's TensorFlow. What we have done is essentially allow them to take some of these pre-built models and then transform them to something that is even more simple, right, sort of, and customized. And typically, you would need a grandmaster who understands how to summarize text, right, sort of, you have the BERT models, but you still want to kind of train it on your local data and attach and stitch them. I think that's the place where a lot of our customers

either rebuild everything from scratch or they need to retrain. And I think what Hydrogen Torch allows them is a no-code way to take advantage of pre-built models, but also bring the aspect of fine-tuning it to your data. Okay, well, thanks. Again, you foresaw my follow-up question, which was going to be precisely about that.

So my initial impression, I think, is validated by what you just said, that the fact that what one of the things at least that Torch seems to offer is a simplification, let's say, of underlying algorithms in what looks like a no-code environment, which obviously helps people who do not necessarily have organizations who don't have the necessary skill sets on board. However, I wanted to check with you what happens. So how does that support, let's say, the end-to-end life cycle? So from conception, and we also touched upon the fact that, well, even though you may be using pre-packaged algorithms,

typically you need to also retrain them a little bit to tailor them to your specific data sets and so on. So how is that supported in this environment? And then also from that point on, how does the actual deployment happen? And I also have a follow-up question on that, but I think that's enough for now, and then we'll get back to this.

So for models like NLP, where you're doing text summarization or picking up categories and entities inside and image and video, we basically have a very tight-knit integration into our operationalization, the MLOps side. And I think being able to connect the dots into the model validation phase, where is the model drifting because new data has come that shows that the model is behind and you need to rebuild the model or the customer sentiment has changed. So you need to rebuild the model. I think sentiment has changed. So you need to rebuild the model.

I think that's the place where there's a very deep tight-knit integration into the operationalization that H2O AI Cloud offers. So Hydrogen Torch lives essentially as a core engine for AI. It's roughly in Kubernetes, containerized inside our H2o AI Cloud. So as a result, it nicely feeds into the data

and machine learning pipelines that are going to production as either rest in points or embedded inside applications. So that then when the models as either rest endpoints or embedded inside applications. So that then when the models have been continuously being monitored to see if their accuracy has changed, so then because of the data change or because the behavioral change of the end user,

then you start rebuilding the model again and then redeploying it in a almost a continuous learning systems that continuously learning containers that can essentially adapt as the data is changing. And the other thing you had which was a very interesting one I would be amazed not to mention is that the Kaggle grandmasters that we have at H2O, like Philip Singer, who is on this call, or Dimitri Godive, or Eugene Papakin, and Olivier, they basically were the leading kind of winners in the space, which of the wall number, former wall number ones. And so as a result, they have done really good work, whether it's in terms of Peng Xu,

who's a former one that's doing a lot of work in images and steganography. Philip is one of the fastest, world number one in Kaggle, has done work in understanding like audio detection and rainforests and getting some of these top problems that like the SETI and so these top data scientists essentially brought their, you can call it their craft, right, to H2O. And it would be, they would be, typically they would be in some department of marketing or ad tech or trying to help a particular problem

in a particular large company. What they have here is they come here and work to see a general class of problems across thousands of our customer community. And as a result, get to generalize those problems. And some of the insights they have had in winning these Kaggle contests now become generalized, recreated, remade as software, whether recipes or in this case, a whole engine. In the driverless AI, we actually had Dimitri Larko and Marios and Mark Landry and others who are good in time series

come and own that whole recipes for solving time series problems and democratizing that for every supply chain distribution problem to every kind of classic time IID data. What you're seeing here in Hydrogen Torch is Philip and Eugene and rest of, Dimitri and others have come together to solve the class of deep learning problems,

whether it's for NLP, image, audio, video, vision. And so that completes the H2O AI cloud side of things where we can absolutely challenge the incumbents and at least allow our customers like AT&T or Citibank or Capital One or Commonwealth Bank of Australia, take on the tech giants or fintech giants that they're fighting or Rekit or Walgreens. And I think that's the exciting aspect of the co-creation between data science grandmasters and

the core systems teams and math physicists at H2O. Okay, thank you. You did refer to accuracy of those models, and obviously that's a very important parameter. I was wondering if you're able to share any metrics for the models that Torch includes. And then also you mentioned that their accuracy is continuously being monitored and retraining takes place as necessary. And I'm wondering if this, well, first, if this monitoring is transparent to the users, so if they can be aware of how the model accuracy affairs are evolving through time. And then if they're also

involved in this retraining process that you mentioned, do they actually need to, I don't know, reconfigure their pipelines or do something else? Or is that something that to some extent is taken care of by Torch itself? So it's a good question. A lot of our data scientists want to be hands-on, right? And the best ones don't have time, right? That's kind of our vision is to not, to actually empower our data scientists to save time with it to document the models. So we produce automatic documentation of the models

in terms of what was picked, what are the transformations that happened and what images were involved that triggered different end results and so on. So automatic documentation kind of gives a way for data scientists to go back where the great model was there, like almost like a time,

go back in time and look at where the best model is produced and then go back forward and promote the best features that they found before so in driverless ai beaver it is not an atypical it's almost common to have models produce 20 30 percent better like go from 60, 65% accuracy to all the way to 89, 95% accuracy for mortgage underwriting type, really valuable model fraud prevention and customer experience,

CX marketing or supply chain distribution. So it's very common to see models that are world-class, almost relatively out of the box with driverless AI. We are expecting to see almost a very similar leap for hydrogen torch. It's coming out and we are seeing very early good results in the labs. Some of our earliest customers are reporting that they would have spent a very long time to fine-tune that deep learning layers to get that kind of similar high accuracy that they would otherwise get out of the box.

Or with a limited amount of steps of letting the discovery process be in hydrogen torch than regular torch or tensor flow. So I think we're expecting to save a lot of time by getting at least the same or even better accuracy. Of course, in deep learning, accuracy is already in the 90s. So kind of going from 87 to 91, while it looks small, it can be a leap for some of these problems that we're looking at, or for some of the work we're doing with NLP, getting to high 90s means that you can, a cancer referral doesn't have to wait for a manual intervention. And suddenly, and before the tumor is growing faster, you can absolutely refer the patient to the right doctor and boom, you can get

to saving lives faster. So I think in this case, time is the only non-renewable resource and inaccuracy is super important, right? So the coin and currency of our space is speed and accuracy and interpretability is third dimension. How do we take a black box model and then break it down to locally linear models that you can understand and determine why a model was doing what it was doing. I think those are the three big dimensions that we operate on.

And sometimes we trade off speed for interpretable accuracy or we trade off accuracy for interpretability and sometimes you can make the key focus will be just all in on accuracy so if you're a hedge fund or if you're a fraud prevent trying to prevent fraud you're all in on accuracy and you go very deep and if you're a regulated industry where you want to understand the business implications and if there's bias in your model then interpretability takes a little higher dimension but in all the cases time is super important so speed matters in almost every case. Yeah thank you and yeah it's interesting I agree I think everyone who's even vaguely familiar with

that space would agree that those dimensions that you mentioned are indeed very important and there's always some kind of trade of involves when you're choosing what to optimize for and interestingly you also mentioned well speed and I think well to some extent at least that's connected to something I've been meaning to ask you which has to do with the kind of deployment targets that are supported by Torch. So you know your deployment targets may vary widely so from very limited edge devices to super instances in the cloud and the kind of models that are produced directly influence both the ranges of machines you can aim at for deployment and also the speed of execution. So as a rule of

thumb, the bigger the model, well, the slower it executes. And that also translates to operational costs and so on and so forth. So I was wondering, and I have to say that I'm influenced by people in startups that I talk to who actually focus specifically on that, on producing models that are more frugal, more economical to operate. And I wonder if that's a parameter that you're also targeting with Torch. I think a very, very, very meaningful question. So GPUs are the bread and butter of,

for most of, most of the deep learning frameworks today. And I think for training, that still is a magnitude faster, right? For training, deep learning, hydrogen torch obviously consumes a lot of GPUs on the training side. On the scoring side, I think we are able to miniaturize a lot of the models so they can run on the edge, on a drone, on the ARM processors, and CPUs, of course, and GPUs. So I think the smaller the end payload can be, the better, more federated, more distributed

intelligent things we can see. So our vision over time is to democratize intelligence from hybrid on-prem large servers to cloud and APIs from the cloud to actually deployment on the edge and in the field. And I think that continuous kind of long-term thinking for us means that we need to kind of further miniaturize the scoring and training backends. The current model is obviously heavily influenced on being able to take advantage of large servers and speed on the training side, but I and the scoring is on CPUs or cheaper and expensive hardware. But I think over time, that will probably continue to be simplified. One of our customers, for example,

has taken our models and miniaturized them for ARM processors that can run and live on a telephone pole. When you have a drone that is trying to go and assess a home fire, for example, to prevent loss of life and property, it has to react locally very quickly so that that drone has to transform what's happening locally and talk to its local edge server and be able to react

on the spot. Those kinds of use cases will demand us to have even more smaller system. And over time, sustainable AI is going to be a drive for many of our customers, especially as they think about sustainable supply chains, like for example, working with a very large retailer on which stores reopen after COVID and kind of for brand protection or for revenue, you want to have

a sustainable supply chain and you want to also have a sustainable AI. And in terms of power consumption, I think we're going to see a lot more of drive in the coming years on trying to use simpler methods that consume less CPUs, less GPUs, less processing power, less power, and be able to produce nearly similar results in terms of accuracy and interpretability. And I think that's a further continued simplification of our space. Once we get several with no code, we democratize the usage, right?

Sort of the ease of building models, but the making of models, and then ease of taking those pre-built models and then transforming them as data is changing? And then, of course, can I then adapt it to a low-cost, low-power method? For example, we all know mammograms have high false positives and also are expensive. If you then switch to something that is much widely used and available, like for the same techniques that one has for childbirth, if you use the same methods and use deep learning on that, now you're able to kind of essentially detect tumors much earlier. So I think using a less sophisticated data capture, a more sophisticated deep learning model,

you can come up with better solutions for a global world, which is mostly not the richest part of the world. Seeing a lot of equity and inclusion, need for equity and inclusion, given the world being more and more imbalanced at this point. We think that AI has a perfect role to play to democratize not just intelligence at the software technology level, but intelligence up the stack in terms of domain expertise, the world's best

onco or cancer physician or cardiologist or immunologist can be captured. Most of what he or she does can be captured in deep learning models like using hydrogen torch that can then

now be accessible in Africa and India and in places where on the cell phones and one of the problems we were solving for is for midwives to be able to predict difficult childbirths ahead of time and again it's a very big image recognition problem and easily solved. Times 3 is a plus image recognition problem, so it's easily solved with cell phone images from Kenya or poor neighborhoods in Mumbai and Karachi. And I think being able to help empower those midwives with really powerful methods that use deep learning is help save lives. We're working with wildfire prevention use cases where bushfire wildfire image images and predicting which side of flames will be moving in another powerful AI for climate change hurricane prediction one of the problems

we're solving for which telephone pole is likely to get hit by a hurricane for one of our customers. But that, again, also helps the local region to be able to kind of adapt at least a few weeks ahead of time. Endangered species, census of endangered species. We're working with the UNESCO top AI project called WildMe, where we are looking at endangered species and coming up with census of them using,

again, deep learning methods. And I think what all of these use cases are pointing to is the applicability of AI is just getting started, right, sort of you're in first innings of AI and democratizing the methodology of making good AI will allow us to go really far. Okay, so obviously I take it that miniaturization of models as part of this democratization is in your roadmap

and so since we're almost out of time, I guess I wanted to ask you if you could just briefly share in five minutes or less what else is in your roadmap after releasing Torch. In terms of product methodology, I think getting the various sub... So we talked about document AI.

I don't know if we briefed you on it. NLP documents, along with Zoom, audio, podcast, and video documents have been the fastest data source growth that we have seen under COVID. COVID has really amplified these data sources. So we're following the data that is coming in and trying to innovate to allow our customers and communities keep up with that data sources

that are coming in and ability to react and collect new ways of data. And that's where the Federation of AI comes into play. But Document AI is one of our fastest growing new product that we announced earlier this month. And that is led by another incredible data science grandmaster team, Mark Landry and others. What that is allowing customers is to take advantage of the millions of documents that are coming in pages

that they can quickly process and go beyond OCR to intelligent character recognition and combining image and NLP methods. So they can then automate. Typically, an RPA plus document processing system is giving them a rule-based mechanism. Now with AI, they're able to go beyond just rules to decisioning and smooth up the workflows as businesses.

Federated AI is the other place. We think that a billion cell phones every night can be processing the same AI that would otherwise be in an expensive Amazon cloud or a Google cloud. And I think that's the place where we see a very strong kind of future directions where I mean, neutralization,

but also distributed low power AI. And of course, AI is going to touch every space out there. Content creation is a place where we see a lot of AI being used in the like synthetic data creation to kind of synthetic content creation, summarization. People have a lot less time, right? So how do you summarize into brief statements what would otherwise take a long read? I think that's the place where we're seeing sensor fusion between podcasts to Zoom to audio to kind of NLP and time series. I think that whole combined,

bringing all of them together is a place. Telling stories is still very hard. So for us, we think that AI that can tell powerful stories from data or allow data scientists to intervene and tell a very powerful strategic story and narrative will be very important. So the future we see is the data journalism, the same role data scientists played in the last 10 years. I think we'll see the rise of the data journalist in the next 10 years, where every department, every group, every organization will need a data journalist to tell a very powerful story on what's really happening in their space with real quantitative backing. And I think that's a place where we see a lot of emerging innovation.

Lastly, I would say applications of AI. We're co-creating with our customers, with Commonwealth Bank of Australia, with AT&T. We have revenue share-based models where we can co-create with the domain experts, Richard. So sitting down with a very deep economist of a bank, the rise of changing interest rates, what's going to happen, and what are the models that you can build for these scenarios so companies can adapt, inflation is going up, how do companies adapt. I think bringing AI to serve our communities, so AI becomes less of an onboarding problem or a distrusted technology to a very trusted technology. And dehyping that is going to be continuously our role is to build that trust in AI.

We think of AI as reducing uncertainty and fear of the future. We are reducing that anxiety. And that means that we are in the business of making customers happier. Right, to observe both the communities and customers feel happier if they can essentially have less of fear and anxiety and I think that's kind of at the heart of our mission is to make the world a better place and make people in generally pursue happiness and that's possible if we can make intelligence be their true true partner in everything they do well i would say that you have described both grand vision and well a handful of potential products waiting to be

to be made and released and specifically as far as the focus on data journalism goes, I have a little bit of experience myself there. And yeah, I can attest to the fact that, well, it's a long and laborious process. So if you ever decide to pursue that, well, feel free to let me know. And the least I could do is connect you to some people that work in the domain and we'll be happy to test drive a product if you ever have one. We would love to have you, George, in that group of advisors building a powerful data gen as misinformation killed more people than even the virus. So I think the lack of being able to, we are a species at war with COVID. What it has exposed is we took a very localized approach as opposed to a global approach. And I think getting back to what we are really good at,

embrace our diversity while not breaking down into local chambers of thinking is going to be super important. And I think journalism and data journalism is going to play a huge role there. One app we recently released on that is actually for hospitals. I know you have to run, but we just released this application for hospitals to simulate what is likely to happen ahead. And I think I'm calling from Dallas here

and we can tell kind of where the future looks like. Seems like ICUs are going to, stays are coming down, which is a good sign. But in general, I think it is going to be more and more critical for us to use decisioning systems to predict what is going to happen and go back to the sources of the data in one place you want to know where the data came from you want to know like essentially drive the storytelling

with the data attached just like you would have it in an excel when you're going into a boardroom you want to have that level of clarity for stories coming straight from data with the fine tuning from the journalists with that, you can put your fingerprint on this and our watermark and say, I, George, attest that this data is right. And I'm using these tools and this data. These are the source from this. And now you can go forward. And by the way, data has changed. You're going to rebuild again. Yeah. There's, there's many challenges involved,

not least of which, well, actually getting the public to be involved in educating them in a way and how to interpret those things and cross-checking your sources and all of those things. But well, that's a story in and of itself. For the time being, I'd like to thank you very much for uh for your time and uh the the conversation i really enjoyed that and well uh good luck with everything going forward we're just getting started we are so fortunate with the community support for our um our mission and movement and um folks like philip and the

gaggle grandmasters were in the community. They joined us at the moment and then made it bigger and better. So, and journalists are currently in our community. And when I think of a journalism from data, it's really that human AI interface, right? Sort of, if you can make that even more smoother so that just like we're using cell phones,

we should be using data. And I think that level of ease of use can happen if we immerse ourselves into making it really a power tool for humans to excel at whatever purpose they want to pick and then have impact. I hope you enjoyed the podcast. If you like my work, you can follow Link Data Orchestration on Twitter, LinkedIn, and Facebook.