The Data Stack Show - Shop Talk: Accountability and Opportunity for AI

Episode Date: June 23, 2023

The Data Stack Show is a weekly podcast powered by RudderStack, the CDP for developers. Each week we’ll talk to data engineers, analysts, and data scientists about their experience around building and maintaining data infrastructure, delivering data and data products, and driving better outcomes across their businesses with data.RudderStack helps businesses make the most out of their customer data while ensuring data privacy and security. To learn more about RudderStack visit rudderstack.com.

Welcome to the Data Sack show, Shop.Costas. We have talked with people who built amazing data technology at companies like Netflix, Uber, and LinkedIn. But you and I actually don't record our talks about data very much. But we actually talk about data together a ton. And so Brooks had this amazing idea of just recording some of the conversations that you and I have before and after the show about data and our opinions on it. And really, this has been one of my favorite things that we do. So welcome to Shop Talk. It is where Costas and I share opinions and thoughts on a personal level about what we're seeing in the data space. And it really is simple.

We ask one another a question and the other one tries to answer it. So without further ado, here is Shop Talk. Welcome to the Data Stack Show Shop Talk where Costas and I talk shop. Costas, it's been a while since we've done one of these. Lots has happened. I have a new baby in my household, so that's kept me away from the microphone for a little bit. But this is exciting.

Oh, yeah. Yeah, I miss pre-styling with you, so let's do it again. One of the interesting things about the primary interface and ergonomics of chat gpt is that it's one i think it's amazing because it's just so familiar right uh it just such it's actually almost you know 10 years ago it's like you know, 10 years ago, it's like, you know, the most popular, like the most widely adopted, like,

LLM is going to be like a text exchange. You know, I mean, it almost sounds crazy when you step back and think about it. But brilliant from an interface standpoint. One of the things that's really interesting about presenting a bunch of options as search results that have some sort of ranking that's at least nominally transparent

to the end user who executed the search is that you're presenting some level of choice. One thing that's very interesting about the chat interface is that when you put in a specific query, the results are largely singular, right? And so you're actually not presented with a lot of choice. And that really, in some ways, to return to the accountability question, what's really interesting about presenting choice is that it the there's more

accountability for the user because they have to decide right like i'm offering you a variety of things and i'm applying some weight that may or may not be transparent to you but you ultimately have to decide what you believe to be sort of the best match. But it's very linear and you're getting a pretty singular response. How do you think that causes, I mean, I think it caused, that's one of the roots of the issues of accountability, right? Is that it's almost a binary choice. Like, do I trust this or do i not and i don't have there's no surrounding context that gives me a tool set with which to make that decision yeah yeah 100 and also i think it's even like worse than that because even like the same question if you ask like such dpt like it might might generate a different response.

It's not as deterministic, let's say, as it is when you search on Google. In Google, in a keyword, you're going to get the same results all the time. With the chat GPT, that's not the case. And I think it's one thing, and that's where things are getting quite interesting when it comes to integrating it with products and trying to productize it.

Right? Because how do you account for that, especially in use cases where critical decisions are made? Like, let's say I'm a doctor and I'm using chatGT to summarize stuff, how can we build a user experience product that it will help the doctor to do the summarization, but at the same time will safeguard both the model and the doctor from interpreting the wrong information or you know like making mistakes that it's like hallucinating like interpreting like hallucination like for example let's say just like what the like pretty much like everyone i guess like have heard about like

a large language models so i think that's like the interesting part. And like, that's why I'm very excited. And I think like the people who should be like extremely excited about AI and especially LLMs are product managers. I think there's like a very new modality that is given to them, a very new tool, which is almost like completely uncharted territory right now. And I think there's a lot of space for creativity in product design and management to make some

very interesting and create new types of experience there. Yeah, I totally agree. So I almost, this is distilling it down to like two levels, which is not actually reality. But reality is a construct, so it doesn't matter and I'm going to roll with it. Even the Matrix, right?

Yes. I think that there will be, actually I re-watched, that was on in the hospital when the baby was born. And I forgot how good of a movie that was. It was, I remember watching it for the first time. Yeah. The first one.

Yeah. Like, yeah. Unbelievable. Yeah. But so I, one thing that is a little bit disappointing to me about the hype cycle is that you mentioned it earlier.

This is going to drastically accelerate the amount of human creativity that can be put into jobs by doing things like pulling information out of PDFs and summarizing it, right? I mean, you mentioned the medical industry, right? I mean, there are entire like sections of publicly traded companies that have armies of people who go through PDFs and manually like copy information from a PDF into like some other system, right? I mean, that is wild to think about, right? And it's like, okay, those are creative human beings. If we can remove that from them and allow them to attack different problems, that to me, the first wave of the types of things that this will do will actually be pretty utilitarian like that,

and they will just unlock a bunch of bandwidth, which to me is extremely exciting. I know there's concern about job loss, but I'm with you. I'm actually not concerned about that because I think that it will free up human creativity, which will be a net benefit to society. That's pretty exciting to me. But then I think then you know, once a number of those problems are solved, you, and even now what people like to talk about more is the tricky thing. Should this LM be able to make a diagnosis for a doctor? And where is the jurisdiction, you know, from the doctor and from the patient? And how do you regulate that? And, you know, that's where things get really tricky. And that's where people tend to

focus the conversation because they're like, what's going to happen? You know, is this thing going to like wrongly diagnose your disease? But to your point way earlier, there's going to be monumental changes for things that are not that exciting, like from a news cycle standpoint, right? Yeah, 100%. Actually, I think there is a... I keep sharing these insights because, I don't know, it was very surprising for me when I experienced it. But I think it's a glimpse to the future

of how AI is going to be, in my opinion, implemented and how we are going to embed it in our everyday life. So I don't know. I guess everyone is familiar with Invisalign, right? Even if they haven't used it, they probably know what it is. If you go through the process of doing a treatment within VisaLine, you are going to notice something if you pay attention. Okay? The whole process can happen without actually having a doctor there.

Like, you don't need the doctor. Like, you don't. Even what the doctor is doing when you don't need the doctor. Like you don't. Like even what like the doctor is doing when you go there, like at the beginning, uh, it's something that like you can pretty much do like on your own. Like you scan your teeth, they take these scans with a phone, by the way, you can do it, feed it to the algorithm and the algorithm comes up with a treatment. And the treatment is then fed into a 3D printer. They print the Invisalign, the molds, and they send them to you. But no one does that.

We always go either to a dentist or an orthodentist, and we go like through this process and the reason we do that is because the modality is like so so different that like humans would never like trust that right like it will take time to get to the point where you're like i'm going to put something in my mouth for like the next six months that is completely generated by a machine and it is completely generated by a machine. And it is completely generated by a machine. The doctor there is there to take care of like, okay, some cases that the, like the edge cases of the algorithm or like complications, right? And most importantly, to make you feel safe and trust to the process.

Okay. Because you are interacting with a human, right? Yeah. Now, the benefit, did the orthodontist lost their jobs? No. Just that more and more people at the end got access to this kind of treatment in lower prices and they scaled their jobs because now they can, in their offices, they can have 5x

the patients that they had. I think that's what is going to happen with other professions that you have lawyers interacting with the public sector. All these insurance claims.

All that stuff. We will just be able like to do more. And that's going to be like, in my opinion, like a positive sum game at the end compared to the probably negative sum game that like crypto was. Yep. Yep. Yeah, that's super interesting okay one last one last thing what have you built and have you built anything with an flm yet yeah i'm using

i'm using them like a lot i mean i'm using them yeah like i I mean, I'm using them. Yeah. Sorry, beyond just a personal, because I think a lot of people who are interested in it use it for different things on a daily basis by just sort of inputting a bunch of prompts. But have you, like... I think you've built a couple end-to-end things on it, right? Yeah, I'm building stuff. Yeah, like, for example like one of the ways that like, and it's funny because you

mentioned something similar, like for example, I wanted like to build like a synthetic data generator and you know, especially when you have like low cardinality data that you need there. it can be like super helpful because it can create like much more realistic data. So for example, let's say I want to even like with, let's take like the, a rather stack event, right? Like if you want to create synthetic data for the agent field, instead of trying to figure out and putting just a junk of characters there, you can get, I don't know, a variation of like a hundred of them and just sample them and use them to generate data.

And you don't have to continuously do that. It's just to seed your data with that because it's slow to do it every time and expensive. But I'm here to do these kind of things. I'm using it to code. It's super useful,

especially... I was never the person that enjoyed writing tests, so now it's pure joy to write tests using Copilot. I'm doing a lot of... I have used Whisper a lot to analyze the episodes here that we are recording. I'm using MidJourney to generate some assets for presentations and stuff like that. By the way, keep in mind, I can't even draw a straight line. For me, it's a huge enabler that I can go there and create something decent. It's not like it's going to be perfect. It will never be perfect. And yeah, I will need a designer. I will never substitute the designer. But at the end, the designer can iterate much more on what I'm asking because of these tools.

So we can end up in what I need faster. What else have I done? I think these are the things so far. I'm using it a lot while I code, to be honest. It's very interesting. It cannot substitute coding but it changes the experience a lot what has changed is that I'm not using Stack Overflow anymore I mean I'm not writing code in the same way that you are

but querying data a lot I've never had a job where one of my stated responsibilities was writing SQL or Regex stuff. And so if you've never done that professionally, and I just haven't spent the time to actually get over the hump of figuring out how to structure a query in a way that is efficient and whatever, it is amazing

for that where it's like i'm trying to answer a question and i know like the basic structure of how it should look and it's like just corrects this yeah it's amazing yeah the other thing that's also like connects with a conversation about like google for example, they're like, okay, I've never learned or there are specific terminal commands that I keep forgetting exactly what the parameters are. Yeah. One of the biggest problems that I have is I always forget how to do an SCP.

And I usually just go and Google it because you pretty much get the exact answer when you want. But what happens is like people have like games assistant show maths where you are like ending like in a landing page where there is the command. And there's show maths advertisement on it. Like show me like it's so annoying. Like I can't see anything. I know exactly what you're talking about. Yeah, so

now, instead of doing that, I'm just asking the LLM and it gives me the commands. Even if it doesn't work, it's close to what I need, right? But I didn't have to go through all that crap of all these stupid ads. I totally agree. I think also, you know,

Google in a way also heard themselves by allowing this to happen. Because at the end, it's all about the experience. It's the same information. There's nothing creative there. Totally agree, yeah. It's just the experience're really bad on Google. Yeah, it's an absolutely fascinating study in user interface.

I totally agree in accessing information. It's also going to be really interesting to see how this impacts Microsoft and kind of what that does in the market with the big players, which will be super fun. All right. Well, Brooks is telling us we are at time here. And maybe we'll try to, I don't know, we should

try to ask an MLM to generate an entire episode around a topic and see if they get our tone and voice? That would be interesting. We can do some, we can be creative. Let's have like a sub GPT as a guest. Yes. And see what happens. That would be fun.

Let's do that. All right, we'll try it. All right. Well, thanks everyone for joining us for another shop talk. Costas, thank you for sharing all of your thoughts, even if they're, you know, just a construct and we will catch you on the next one. You know, Costas, we learned so much from the data leaders that we talked to, but I learned so much from picking your brain and actually your questions really make me think really hard. So I appreciate Shop Talk. I think it makes me a sharper thinker.

Well, it's fun. Like, I think it's good to just sit and chat about the stuff that we experience. And yeah, I think like I hope like people enjoy it. That's why I'll keep asking for people to reach out.

Please do this. Come on, Pog. Like you can do that. Like send an email. Yeah. Let us know how you feel and like, what are your opinions of like your experience with the show? So please do that.

So me and Derek, we can keep being happy. Of course. And of course, we try to take the same types of questions to data leaders from all sorts of companies, large and small. So definitely subscribe to the main show if you haven't yet.

Tons of really good episodes there and tons of really good thoughts from data leaders, you know, really around the world. So definitely subscribe if you haven't and we'll catch you on the next Shop Talk.