The Data Stack Show - 192: Business Logic As Code: A New LLM-Powered Operating System for Business Automation with Binny Gill of Kognitos

Episode Date: June 5, 2024

Highlights from this week’s conversation include:The history of computer science and AI inflection point (1:23)Binny's early programming experiences and the constraints of technology (2:14)Getting interested in computer programming (5:02)The experiment that impacted the starting of Kognitos (8:23)Challenges in traditional computer science (16:04)Reimagining programming and debugging through natural language (19:08)The operating system for applications (20:19)Changing the paradigm of programming (21:25)Complexity in software compilation (22:05)Challenges in automating business processes (24:50)Solving business process problems with Kognitos (27:39)AI as a tool in business solutions (34:05)The future of AI and specialized intelligence (37:08)Using LLMs for Context Generation (40:43)Biases and Data Set Source Transparency (41:48)Next Innovation in Data (44:34)Final Thoughts and Takeaways (47:06)The Data Stack Show is a weekly podcast powered by RudderStack, the CDP for developers. Each week we’ll talk to data engineers, analysts, and data scientists about their experience around building and maintaining data infrastructure, delivering data and data products, and driving better outcomes across their businesses with data.RudderStack helps businesses make the most out of their customer data while ensuring data privacy and security. To learn more about RudderStack visit rudderstack.com.

Hi, I'm Eric Dotz. And I'm John Wessel. Welcome to the Data Stack Show. The Data Stack Show is a podcast where we talk about the technical, business, and human challenges involved in data work. Join our casual conversations with innovators and data professionals to learn about new data technologies and how data teams are run at top companies. Welcome to the Data Stack Show. We're here with Benny Gill of Cognitos. Benny, thank you so much for joining us on the show today. Hey, Eric. Thanks for having me

in the show. All right. Well, give us tons to dive into, but give us just a brief background. Where'd you come from? And just a little bit about Cognitos. Yeah. So I'm a software engineer by profession. I've been writing code for 30 years. I started Cognitos about four years ago. And prior to that, I was CTO at a company called Nutanix for about eight years grew from zero to an IP and beyond great experience there learned a lot my

experience prior to that is an IBM research mostly in the storage background bringing cash technologies to the enterprise masters and from UIUC and bachelors in IIT Kanpur computer science. Grew up in India. Great. So, Benny, one of the topics I'm excited about talking about is the history of computer science, how that's evolved, and then this AI inflection point that we're at now and how things are changing, and then some really unique ways that your company's

trying to solve those problems. Yeah, that's a topic very close to my heart. When I was a teenager, I was programming and I had a handheld Casio graphing calculator, which happened to also support basic programming. And I had a large total of four kilobytes of memory where I used to fill in my basic programs. And I had to be really careful about how much code I'm writing.

One of the first cool programs I ever wrote was a tic-tac-toe game. And it was easy. It was like lines and Xs and Os. So it was doable. And I showed it off to my friends and they were like, what is this? I said, no, this is a computer. But this is not an equation. no, this is a computer. And but this is not an equation. You know, no, this is basic language. Those were the days, there was no internet, there was no YouTube,

I just had a manual. And I love the power of making a machine operate in a way that was custom to what I wanted. That was the power that, you know, got me hooked. Until date, you know, that's what keeps me happy with what I'm doing. Yeah, that's exciting. We're excited to dive into that. All right, well, let's dig in. Benny, I love the story that you were telling us when we were prepping for the show. Your father recently gave you a notebook from your childhood and on one of the pages you wrote 975 kilobytes right then yeah so great so you wrote that at the top of the page so give us the story like

give us the story of the notebook. What was the notebook for? And your father, you know, dug it up after so many years. Yeah, so it's a physical notebook. And I had almost forgotten about it, but my dad preserved it. That was a notebook of all the programs and basic that I had written in this handheld Casio calculator slash computer that I had. And I forgot why I had written it.

Normally, people don't write computer programs in a notebook. And I saw the number on the top and it said 975 characters. And then I realized, characters. I mean, because it's one page, so you can actually count number of characters, right? So, the reason I was counting the characters, and I was also writing it

out, because back in those days, there was no internet, there was no connectivity between machines. The only way I could create room in my computer was actually to delete stuff, but I didn't want to delete programs that I wrote with a lot of effort. So I would actually jot down the program on my notebook, but then I also would write number of characters it frees because now I can write

another program, but it has to fit in that. That is one of the first air-gapped backups, I think. Yeah. Very secure. You know what? It's still backed up. Every bit is intact after 30 years. That's a solid track record.

Maybe even some foreshadowing for what you did as a career, too. Yeah. Where did you get the calculator, and how did you figure out that you could write programs on it so the story is that so my dad was a mechanical engineer by profession he would do engineering drawings and back in those days computers weren't a thing right so the best job you could have is you're designing the machines that will be built in a factory. A factory is where molten iron is being poured and all sorts of things. And you're sitting in a nice clean room and you're designing stuff and optimizing things.

And one day my dad comes and says, Benny, you should not grow up and do what I'm doing. I said, oh, what happened? This is such a cool thing, right? I like to design stuff and build stuff. And engineering is sort of what i liked is you know i saw autocad today in the office i said what is that oh it's a computer and i saw that in two minutes somebody could build a drawing and print it out with a big plotter

and what somebody like my dad does in a whole day you could do it in five minutes said that is the future and so you should do computer programming or whatever it is and there was no computers out there i had never seen one my dad saw one in the factory and like was blown away and after a few weeks he comes home with this Casio personal calculator slash computer, and he says, this is a calculator, but it also can understand some computer language called BASIC. So here's the manual. Go figure it out.

That's how it started. Wow. That's cool. I think it's really, and John, interested in your thoughts on this, I think as a parent, i have so much appreciation for your dad not reacting to what he saw and fear but seeing future opportunity for his kids you know yeah that's just such a that's so encouraging and your dad sounds like a great man and you know and i was and i didn't know what to do. Like, what could I do? Nobody knew. Like, I started off drawing lines because I had this idea that AutoCAD draws lines and I got to draw lines.

And I said, but what can be nice? And then I realized tic-tac-toe is nothing but lines. So let me just make a game. So I did tic-tac-toe and showed it off to my friends. And like, whoa. Video games were becoming popular. Heavy, very expensive pieces of equipment to buy. And yeah, but here I could make my own game. And that was, you know, that got me into computer science.

I learned a whole bunch of languages in the three decades after that. Yeah, I'm trying to follow that as well. Well, so the second part of the story, for a Casio calculator that could also do programming. And the revelation was that your son wasn't any faster than you were, even though he had YouTube and Khan Academy and all these amazing resources at his disposal. Yeah, that was the moment where I decided to start my own company and start building Cognitos. What happened is it was the pandemic and the schools, public schools were closed and kids

were like, we don't know what to do. The schools hadn't figured out how to do curriculum remotely. My son, 12 20 years old he was getting bored at home and i said you know what you should learn programming learn python right and he didn't say anything but after a couple of days comes back and shows off tic tac toe to me i'm like this is awesome and i played it it was working was working. I'd say, how did you do that? He'd say, oh, Google, YouTube, figured it out. I was very proud as a dad. Yeah, sure.

And I slept. And when I woke up in the morning, that's when I remembered, hey, I had made the same game 30 years ago. Yeah. Yeah. I was computing. I was the same age, and there was no internet.

How long did I take? I was super excited back then, and I also did it in two days. And I went back to my son and said, you know what? I remembered I had made the same game without the internet in two days. He said, oh, now you're saying I'm not as good or what? No. And then it struck me that something is wrong.

Because you know what my dad did? He gave me a tool that allowed me to do stuff that he would do in a day in five minutes. And he gets me asking my son to do the same thing I did 30 years ago. And it takes the same amount of time. Something is wrong. I didn't believe it. I said, no. Why did it take you so long? Let's go and write a program together, all right?

So I said, I wanted to do something quickly. They said, do you know how to figure out what a prime number is? You say, yeah, of course. You divide by factors, and if it can be divided, it's not a prime. I said, great, we'll write a Python program in two hours or one hour, right?

And then I want to go for lunch let's just do it we sat down and i gave him a python book say no internet because nobody had it back then here is a book yeah i admit it's much thicker than my book my book was only 100 pages yeah now it's this massive i said maybe you just need the first few chapters. Don't worry about the other stuff. All right, let's write. Total blank. What do I do? And I said, first think about a plan.

And I taught him pseudo code. He said, OK, use this right in English. What do you want to do? Computer's not going to understand it, but it's for you to mentally prepare for that thing. OK. Like an organizational process. Yeah. Organizational. Yeah. He's good at math.

So he said, okay, let me write down. So if it's one, it's not a prime. That's the convention. If it's divisible by any number from two to the square root of that number, then it is not prime. Otherwise it's prime. That's what he wrote. Yeah.

I said, that's correct. Very nice. Now let's translate into Python. So first line, if it's one, it's not prime. Fairly easy after five minutes of digging into the book. The next one, divide by factors from two to the square root, involved the loop. And loop is a very foreign concept. And even though you can say that, yeah, you do it for each of these factors, but stuff changes now. I had to introduce variables. I had to do all sorts of things.

It was getting more and more complicated. And then I was pushing hard, because I was running out of time. Like, if it's going to take too long, something is wrong. So anyway, we ended up in a fight. And he said, no, this is bad. I don't want to do it. And I said, no, wait. Let me show you BASIC probably.

So I dug up BASIC and a program in BASIC for prime numbers. I said, do you understand that? He says, no. Okay, shit. Now what to do? And by this, you're way past lunch. Everybody's hungry.

I'm like, this is not going to work out. I said, you know what? I'm going to build a programming language that will be easy for you to understand and all that. And he's like, inquisitively looks at me and says, but why? I said, meaning what? He says, Alexa can already do this. He was pointing to the pseudo code. I said, Alexa can run this. And I said, no, Alexa cannot run this. And he didn't

believe me. And he said, no. In the kitchen, we talk all sorts of things to Alexa, and it's working. So why are you telling me that I need to learn a weird language, this English that I wrote should work. So you imposed constraints where his practical experience didn't see a constraint. Yeah. And I'm like, maybe he's right. And I told him, OK, don't learn Python. Let me try to figure this out.

Because if Alexa wasn't there, then I would have said no. All my computer science training in school has said this is not possible. But here he's saying one line is possible in the kitchen. If you can make one line, two lines in computer science, two becomes ten and ten becomes a million very quickly. We have seen that with all sorts of things in computer science. Zero to one had already happened and now one to two needed to happen, right? And say, okay, I'm going to take this English that we wrote and I'm going to make this work. That's what I did the next three months. I wrote a compiler interpreter for just those three lines to work. Obviously it was a bunch of hacks. I just wanted to see what comes in my way.

Right. Not much came in my way, you know, surprisingly, because deep learning was in a good spot, compilers, you know, parsers, I understand. And I showed it to him after three months and he said okay good he said now can I make a game with this I said oh no a game is hard but then I said okay I'm gonna work on it give me some more time and I really in earnest started building that a proper framework for understanding natural language and then i had so many insights and until date it's like what if i have to summarize one thing is basically it's all about unlearning computer science all of it and only then you'll figure out

really how the human brain works and that's what we need to mimic in machines and obviously we're sitting in a world of llms and people understand understand that now. Back then there was no LLMs. Yeah. I love it. Well, so two follow-ups. One is an observation that maybe more people should look into having their child be their product manager. It sounds like your son, you know, really pushed on it. Yeah, exactly. But two, tell us about Cognito Snow. And so what an incredible story for you to have that experience where your son saw past limitations of traditional computer science as expressed in you know code languages and the way that we create logic for computers to read he saw beyond that you went and solved

this problem and you started cognitos so what is cognitos and and what core problem do you solve or what's the core of the technology and the solution that it provides? Yeah. You know, when I started thinking about it, I was getting worried that this is going to be continuing to be a dark art. Like we are living in, even today, we are living in the dark ages of computer science,

meaning just 0.5% of the world's population can actually read and write computer programs, computer 0.5% of the world's population can actually read and write computer programs, computer language. Wow, that's a small number. It's a small number, yet all of the world depends on computers. Right, right. That's crazy. That is the classic definition of dark age where a few people wield the power of controlling the world. And that's why I'm sitting here as a software developer in Silicon Valley, and last company

IPO and all of that. Why? It's not because I'm smarter than other people who don't know this language. It's just that the language is the key to a lot of power. I was thinking, can we... I mean, we have to change that. Now, why has it not happened? Last 70 years, we have been figuring out how to make computer languages easier and easier.

I spent a full month of February reading up on the history of computer languages, just trying to understand, is there a trajectory where actually it's getting better and at some point it will actually become democratic, everybody understands it. And my realization was now we went from punch cards to assembly and then symbolic languages, a big jump,

C++, Fortran, COBOL. But then now we are in circles. The most recent languages, Golang and Rust and all of that, look closer to C language than actually. That's true. So we are going in circles. And I'm like, oh, this is not going to change anytime soon. Now, the whole world depends on computers.

Very few people have the interest or the capability of actually dealing with computers in that language. Something has to change. Alexa is changing it. Can we jump into this other level? That is what I wanted to go and solve. And Cognitos is a way of bringing it to the market in a way that makes also financial sense

and making it real. I realized that it isn't just the language though. Natural language obviously everybody understands but that's only 20 percent of the puzzle and this I realized after building a prototype and playing with it. And then I realized, you know what? The biggest problem with programming is debugging and maintaining and fixing issues. If you think about Oracle Database written decades ago, you would think by now you should not need any engineer on it. Because you know, it's all settled.

No, there are more engineers now than they were there in the first year. And the code of the database has been bloating and bloating. That's the fundamental problem. And I was trying to grapple with it. Like I had to forget computer science to figure out what the true solution would be and one day you know it came to me grandma's recipe for

apple pie is a program right it's step-by-step instruction and outcome something that program has withstood the test of time. Nobody has filed bugs. It did not have to be, you know, complexity did not grow. If it was a software program, then the first time the oven didn't work, you see, I think you guys already, you know, if it was a software program,

by now it would be a million lines of code. Yeah, definitely. Right, and it would be a million lines of code. Yeah, definitely. Right, and it would be a seven-course meal. And then there would be a section on how to use the fire extinguisher. There would be a section on how to go to a grocery store and get sugar if you're missing that. There will be a section for everything. That is computer science. Now, language wouldn't help there. So if my recipe was in

English, but all these other things were also in English, still it would be a million lines of English, hard to maintain. And then there would be sections that don't match with each other and contradict each other. So light bulb moment was the platform on which the application runs has to be differently built. So the operating system is where applications run. Traditionally in computer science, operating systems do not handle all the edge case scenarios. That's the part the application is responsible for it. Whereas in the human brain being the operating system, if my oven is not working, grandma doesn't have to write it in the recipe. My operating system here will figure out, oh, maybe electricity is not there or gas is turned off or whatever, and I'll go and fix it and come back to the program and run. So fundamentally, there was a need for an operating system that

keeps the application code simpler by being smarter about the world. And nobody had built that in computer science. And I started building that. And that's why it's called Cognitos. It's an OS for cognition. The idea is that build a platform which, yes, can run programs written in English, but more importantly keeps that English simple because the platform is getting smarter and smarter over time. And that's where AI comes and helps. Business logic, as we call it in computer science, we said, this is business logic,

not translated into something. No, business logic is code. And that is the dream. So that's precisely what we've been doing, trying to change the paradigm of how business apps are written. And eventually, it will change the paradigm of how we program computers. You've got me thinking about this, where what if a software compiler erred on complexity? What if there were built-in things to the process of

this technically works, but it's too complex, or this technically works, but it's unmaintainable. Right? That's fascinating. That would be so nice, because isn't that what humans would do? So for example, before computers existed, businesses would do programming anyway. They would have a partner onboarding program. They would have a program for organizing

the end-of of quarter activities. Now, people would write programs as standard operating procedures in English, or maybe employee handbooks would have all sorts of programs in them. Or this is how you apply for vacation. These are all programs. Now, some human would read it and say,

hey, you know what, this is too complex, make it simple. There's always that thing. Now, I envision a future where that standard operating procedure that employee handbook is the final program you're not translating this into python anymore or anything this runs natively on a platform that understands natural language but what's more important, standard operating procedures just like grandma's recipe, don't get polluted with all sorts of edge cases. So the platform needs to be smarter so it can handle the edge cases separately. And then that's how humans operate. So this is

the future is all about creating a paradigm where you can program in a more natural human way. And obviously there is a role AI has to play in there where you need to use AI and yet not give up on the benefit of computers that computers have. John, I have a question for you. And Benny, I want you to tell us how accurate you are

in thinking about how Cognitos could help a business. But John, you ran, so a CTO, I mean, you actually, like, you were a CTO, so you managed all the data infrastructure, but marketing also rolled up to you, which is really interesting. You had a ton of input from the sales side of the organization.

We were just talking about sort of managing for business processes, to your point, right? I mean, you oversaw probably, like, whatever, you know, 20, 20 50 depending on the organization 100 standard operating procedures sure sure yeah so just hearing what benny said like what would you do if you could essentially like operationalize those standard operating procedures that were probably like confluence docs or whatever you guys use like what problem would you solve first if you could

essentially turn that into a computer program so it's interesting so we because we didn't you know have anything like this available we were doing kind of the opposite of we out like for example we had a sales manager that actually learned to program right it's really unique learned a little bit of python and learned some sql and started writing his own reports he had a financial analyst that learned sql so we kind of went the opposite way which is much more difficult right oh sure well that's arguably like not at a point not the best use of that person's time. Right, right.

Especially at a certain level of complexity. And then try to, whenever we made purchases, like, usability was always the number one thing. Like, it's easy, like, especially as a CTO to, like, prioritize features or, you know, other things. But we always basically left it up to the business users for the final decision. You'd have kind of a vetted, here are the options. It wasn't limitless options, but really leaned on them to pick and to own as much as was possible with the solutions. Most of these were SaaS solutions at this point. But yeah, to answer your question,

there are several solutions that we looked into, like, oh, that'd be so great if we could have this business logic apps or workflow or this, that, or the other. And the complexity was way too high for non-technical users and, quite frankly, sometimes for technical users to get value out of it it and you just ended up with practical like you said I mean just like a knowledge base with how-to articles is really what you end up with and when you do that you do get the advantage though of that you know older school way of like when you bring people on and you train them, you get a really unique advantage of making the process better than where something's fully automated.

You don't get that advantage because if it's fully automated, people are like, you know, this thing runs and it spits out this result and we use it. And then they, and then people will go for so long with workarounds in that state because,, well, it's automated and IT's busy and we don't want to bother them. So we're going to do workarounds. And they do it for so long. And then eventually it gets to a breaking point. And then often you have to completely... You're so far away from original intent and maybe different people are even there now.

Then you move to this state of like, okay, we're basically going to scrap that and rewrite it. Mini, help us understand, okay, so Cognitus comes into this world. Can you help us understand on a very practical level for someone like John, where does Cognitus fit in and how does it help him solve that problem? Yeah, so what John said is precisely what's happening everywhere. So before

automation is done, people on the business side know the process because they do it manually. Now they do it in an ad hoc manner. It's not really recorded in a proper way and all of that, but still they know it. Then comes an

automation tool. They say, hey, we could do it, but it's know it. Then comes an automation tool. They say, hey, we could do it, but it's quite technical. Either train your own people who understand the business logic or just write down the business logic for me or maybe we have a meeting. And then there'll be a developer listening to that. The developer doesn't understand business logic as much. But developer will take that and translate it into the dark art. And it goes into the dark art. Okay.

And it goes into a black box. Which is most often Apex. It's not. It's worse. Right? It's so worse. Yeah, whatever. Yeah.

Now what has just happened, we have disenfranchised the business decision makers from actually making changes in how the business works. Because you've taken it and translated it into something that's a black box. Fine. Now it works good for a few months because that's doing what I had just explained to you. Now as John is saying, I want to change something, IT is busy.

And after some time, IT forgets what it was truly meant to do. That's the challenge of having of using a language that is not common between the machine and the business user, right? Imagine a new world cognitus comes in and says, Look, you write a standard operating procedure in your own language. And nobody's going to translate it into something else. That is the program. So anytime you come and you can see what the program is, machine is also trying to read that, understand it.

If the machine has a question, reaches out to you, hey, I was trying to do this. In this particular case, I could not see the discount code. You had mentioned it's in this table in the database. What do I do? You come and say, oh, in this case, just use 10%. And I do you come and say oh in this case just use 10% and the machine says is that just for this case or all times when I don't see this is the default you say oh this is default now the machine has become smarter you didn't program

and your standard process still remains the same it's always readable for the business side. It never becomes a black box. That is the new world that is emerging. That's the correct place to be in. I'm trying to bring the world to a place where computers are, you know, an ID is sort of not visible anymore. Right? So instead of, before computers existed, John would have gone to an intern and say, do this. And the standard operating procedure would be in English, you just hand it over to them. Anytime you want to change your behavior, hey intern, show me standard operating procedure, scratch,

right, and you know, boom, you are programming the human. And that's what machines need to allow people, and suddenly everybody who understands business becomes a programmer even though they don't call themselves programmer I think they are the true programmers if you think about it people in IT developers are not the true programmers it's actually the

business people who say I want this to happen and if this happens then I want that to happen the product managers are the programmers. The actual programmers of today are translators. We don't need translation anymore. That's the point. Yeah. So this is a huge vision. Where are you starting with this vision?

What problems are you first starting to solve with this new paradigm? Yeah. So we are going after financial processes like invoice processing or purchase orders coming in, reconciliation of payments, anything that's document heavy, even if it's shipments and billing, bills of ladings, packing slips, all of that.

Any place, you're smiling, yeah, every business has this problem. Yeah, lots of time in the distribution and third-party logistics space in the past, yeah. Right. So we're working with large companies, Fortune 500 companies, who have this problem at a monumental scale and because machines have not yet been able to solve this problem because it's never cut and dry there are so many variations that only humans can handle now with ai ai can handle variations but you still need a deterministic documented process that is visible and auditable by the business side,

where it's not being translated into something crazy that business doesn't understand. So that's what we are solving right now. We've been doing business in production for more than a year now. Benny, I have a question for you. And I want to dig in a little bit to the AI side of things, because you mentioned AI and LLMs. And I want to start this with, we talked about this when we were prepping for the show, and there's a quote that came to mind. We didn't talk about this, but there's a great Mark Twain quote that came to mind. And Mark Twain said,

God made man in his own image, and man being a gentleman returned the favor. Which I think is a really great quote and kind of encapsulates what you pointed out, I think, very, which is a very salient point in that we're essentially treating AI that way. So I'd love for you to sort of first talk about, maybe react to the Mark Twain quote in terms of the way that we're treating AI, and then help us understand how AI fits into Cognitos. Because one thing that I think is really compelling to me, and I think will be to our listeners is that we haven't really talked about AI this entire conversation. And we've talked about natural language. We've talked

about programming, but you paint it in terms of an operating system where it seems like AI is an input. So would love to break that down. John, would love your thoughts, but start with, you know, start with, you know know man creating god in his own image uh per mark 20 yeah the reason you know i think ai is the tool and we we are in the business of helping people solve their problems and remove their pain points ai is everywhere in what we do except we don't it's not about the hype of ai it It's like what are you doing? So we are in production, in financial processes, things are going on, customers know that the system won't have hallucination and biases and all of that. So talking about

AI is not really the goal. It's about, it's like electricity. When electricity came in, I don't go and say hey I have I have electricity. No, I say, I have a microwave. Oh, I have a light bulb. That's what Cognitos is about. AI is obviously a given. How are you going to use it in your business so that your business doesn't catch fire? Where is the fuse box? Where is the insulation around the wires?

That's what we are the electricians of AI, if you think about it. We're bringing it to the world. Now, the current hype around AI, and Mark Twain, obviously, very smart person, and the way he put it is also politically correct and kind of. But here's what happens. My observation has been humans, any time we have something fuzzy and we think it is going to be powerful we just think it will be like a human right and i was a kid in the mechanical world it was a giant

robot right okay it'll be a giant robot that i could control and it will stomp on us you know a city and all of that that's how i dream about stuff you know mechanical stuff now we are talking now we were talking about ai oh ai will be like a human right agi will be just like human but much more powerful it'll have emotions like gpt4 oh now has also emotions right we were mimicking humans but now look at at reality. In the industrial age, we didn't build robots, giant robots. If I look out the window here or look at my home or my office here, there is nothing that mimics even fingers of a human, legs of a human, nothing of that sort.

But we have machines all over the place. A car does not have legs. It has wheels. Why? Men, wheels are better. You can't just build a machine that runs on legs and goes 100 not have legs. It has wheels. Why? Man, wheels are better. You can't just build a machine that runs on legs and goes 100 miles an hour. Right.

Yeah. Fun sounds. What's that? Yeah. Right? So why do we want to limit the power of the machines we create by mimicking human biological constraints? learned that in the industrial age a bullet train goes is like a million times more powerful than a human okay but doesn't have legs and cannot even

twist and turn who cares right an elevator does not have arms to climb up ropes. I mean it works differently but it's far more powerful than a human. Now think about AI. AGI is like okay I want to mimic, create AI in my own image just like Mark Twain said. No, let's go create artificial specialized intelligence, something that works beautifully for my finance department, for my legal all of that now build a system where the human is in charge human first AI future where just like I get into an elevator and I press a button and elevator obeys me so I'll use an AI system that does finance whatever but it obeys me and And it can only do finance. It cannot have, it won't blush when I say something.

I mean, it doesn't matter. For that future to work, you need to have a platform where you can say, I have a plethora of LLMs or a plethora of AI models that I can use. And now, as a human, I'm stitching these things together, just like on a daily basis, we get into a car, we get into an elevator, we are leveraging different machines to accelerate what we do. We'll be leveraging AI, different types of AI, and we mentally know

which one is safe to use, which one isn't safe. Like when you get into an elevator, you don't see where it's going. When you get into a car, you do see where it's going. So mentally, we need to understand which AI model is going to behave in what way. That's the future, more pragmatic future. And I think that will anyway happen. Humans know exactly what they want. You see self-driving cars, the ones that don't put human first, get recalled. The ones that don't put human first get recalled. The ones that put the human in the steering wheel are still running. So that's what's going to happen with AI, I believe, and I'm pretty excited about it. I was thinking about this.

There is, I call it the GPS theorem for AI. It's like between generality, power, and safety, you can only pick two. So if you're building a general AI or even a general mechanical robot, it doesn't matter, then between power and safety, you can only pick one. One more is remaining. So if it is general and powerful, then normally it's a weapon. Whether it's a mechanical system or an AI system, it doesn't matter.

Wow. If it's general and safe, then yeah, you need to limit the power. And that's what my request is for any people working on AGI, fine, build it, but you cannot make it more powerful. It's like in my home, I will not sleep well if there's a robot that has full freedom to go around anywhere, anytime in the house, you know, and then there's a knife on the kitchen table. Yeah, yeah, yeah. But at the same time, if there is a Mickey Mouse-like hand that comes out of the hood in my kitchen and does my cooking,

but I know it is constrained, it's not general, it'll only do that thing, it can't reach my bedroom, it's okay, that arm can be far more powerful than my arm. And I'm okay with that. Yeah. Yeah. Yeah. I love it. I think, John, I was going to ask you, you know, John and I actually have worked on a number of LLM flavored projects together. And you've gotten much deeper into the guts of it. But Benny, one thing that you said that I actually hadn't really thought about, but when we're using the generic, you know, sort of, you know, like a GBT or, you know, sort of basic like prompt-based,

most of the time is spent trying to infuse the system with context. And so it's pretty compelling to think about an operating system that has that built in. Just interested, John, in your reaction to that based on your experience, because I mean, you're using LLMs to generate prompts that give, it's like, it's sort of an inception level, like using LLMs and multiple loops in order to generate context, because it's so, it's actually very hard to imbue that. Right, which is basically accounting for that it is general, and actually very hard to imbue that right which which is basically accounting for that it is general and i want it to be specific so exactly what you just said

like you spend all these all this time just to get you know a summary of a document or whatever you're working on right of the like i want this to be yeah specific and technical and not flowery and you know x y and z and and then the end result is like well i don't want it to be general i mean i do want it to be general but not really like my end result is it needs to be specific the only reason i like that it's general is the same reason that you liked like that the casio you know where you could like i can code it to do what i want to do i still want that but but the specific is what you end up needing to do anything practical. Yeah. And I believe that

the LLM vendors need to publish what are the biases, what is the data set source for these LLMs? It needs to be open book. And that will do two things. One, we will know what to worry about, what not to. That's one.

The other thing, it will create a future where there is a demand for a lot more variation in LLMs. So for example, if you have to hire a human, you don't hire a general human and then train them to be a good content writer. You interview people and say, okay, this person's content actually I like already. So I don't need to do prompting, it's already there. Now where is the resume of these LLMs?

Because you first look at the resume, okay this is the education that this person has had, they've gone to Harvard or they've gone to this, so that gives you some idea of the data set. And then you come and say, and then in the interview you do this decision which one you like, more importantly humans are consistent. So if they have certain kind of bias they will be consistent, so you know if I'm a democratic politician, I want somebody with democratic bias. I don't want an LLM that could be either or

depending on the prompt and whether I was right or wrong. So fundamentally, the future is going to be a future with large number of LLMs to pick from. Lot of them specialized to the tasks. Our job as humans is just like we interview other humans, interview the LLMs, pick the ones we like for our business, and then fire them if they're not good, get the other one. And I think not much is going to change in how we do work.

We just need to go back to a world where computers didn't exist. And then some humans are sort of like computers. I mean, it's like some humans are like Spock or Data from Star Trek. They are very logical, but they can also understand what you're saying. That kind of power is what we want to bring to businesses.

I love it. All right, Benny, we're close to the buzzer here, as we say. Interested to know, what are you, so outside of the world of LLMs and AI, when it sort of comes to the world of data, what's maybe another technology that's been really interesting to you over the last couple of years? I mean, you've done so much work. I mean, we didn't even get into your work with hard drives and computing speed. But outside of LLMs, what excites you that you've seen? See, I spent 20 years in the storage space

making sure not one bit is flipped from your data. So all the data scientists have lived on technology that I've been working on. And we have gone through a world where we are leveraging data to generate insights. Google is same thing. A lot of data you're generating insights for people. And I think the time has come for the next jump. and this is where I am interested right now. Once you have insight, what do you do? You act on it.

Now, AI can act on it. A programmer can act on it. How do you deal with that situation? So what we believe is data will lead to insights. The real innovation right now is AI can come up with a plan of action based on the insight. But that plan of action has to be reviewable by a human. And therefore, the language has to be non-programming, non-API. That's what we are trying to solve. And that will really accelerate actions based on data. Imagine a world where,

yeah, the AI is going to come up with a plan, but the plan is in Python. I mean, OpenAI is already doing that. But who's going to trust that thing? The first time there is a bad action, you say, oh, I need to hire a Python developer to look at everything that AI is going to do. And there you go. Yeah. And you can't fire the human, right? Like in the past, it's like, oh, that developer no longer works here.

Like that makes it really work. And then you have to fire the LLM. But unfortunately, even that's not happening. We are going after a sort of LLM washing of the world, like, okay, one LLM for everybody. You can't even fire that. There needs to be a Darwinian evolution of ideas inside LLMs as well that's the power of the human race you know 30 years ago what was

considered okay is not no longer considered okay today is that going to happen with LLMs and the only way to happen is right now there there are 8 billion biological LLMs, like humans out there. And they constantly fight to see who is going to win the Twitter war or whatever. Nothing of that sort is happening between LLMs. It's like, I have generated a model spec. Let us align on humans, all humans on that model spec. And that's going to be the future.

I highly doubt. Yeah, I agree. Benny, this has been such a fun conversation. I really can't believe we've been talking for almost an hour. This has been great. We would love to have you back on to dig even deeper into sort of LLM theory and how we handle that as a society and the way we build our technology. But this has been absolutely wonderful.

And thank you so much for giving us some of your time. Yeah, thanks for coming on. Thank you, Eddie. Thank you, John. Great talking to you. The Data Stack Show is brought to you by Rudderstack, the warehouse native customer data platform. Rudderstack is purpose built to help data teams turn customer data into competitive advantage. Learn more at rudderstack.com.