Everyday AI Podcast – An AI and ChatGPT Podcast - EP 140: How AI Will Transform The Business of Law

Episode Date: November 8, 2023

What's going to happen with lawyers, attorneys, and the business of law with AI? Will AI help to improve the role of lawyers and how law is conducted? Enam Hoque, Consultant at LawBeta, joins us to discuss AI and the future of law. From the perspective of those in law and those affected by it, we cover a variety of areas.Newsletter: Sign up for our free daily newsletterMore on this Episode: Episode PageJoin the discussion: Ask Enam and Jordan questions about AI and lawUpcoming Episodes: Check out the upcoming Everyday AI Livestream lineupWebsite: YourEverydayAI.comEmail The Show: [e-mail] with Jordan on LinkedInTimestamps:[00:01:18] Daily AI news[00:04:15] About Enam and his law career[00:06:45] About LawBeta and AI[00:11:05] The future of law and LLMs[00:13:40] How AI can help lawyers[00:17:11] Issues with ChatGPT hallucinations[00:22:30] Ways AI will change the law landscape[00:29:00] AI vs professional law[00:33:00] Enam's final takeawayTopics Covered in This Episode:1. Future of AI and law2. AI's impact on lawyers and the law industry3. How LLMs can be used in law4. How AI and law will impact everyday peopleKeywords:law, lawyer, attorney, ChatGPT, business, business of law, professional, LLMs, hallucinations, AI, GenAISend Everyday AI and Jordan a text message. (We can't reply back unless you leave contact info) Start Here ▶️Not sure where to start when it comes to AI? Start with our Start Here Series. You can listen to the first drop -- Episode 691 -- or get free access to our Inner Cricle community and all episodes: StartHereSeries.com Also, here's a link to the entire series on a Spotify playlist. 

This is the Everyday AI Show, the Everyday Podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business, and everyday life. Meet Firefly AI Assistant, now live in Adobe Firefly, the All In One Creative AI Studio. Just describe what you want to create and the assistant handles the rest, orchestrating multi-step workflows across Photoshop, Premiere Express, and more in one conversational interface. You direct the outcome. The assistant accelerates execution. One thing I always think about is what's going to happen now with lawyers and attorneys and just the law in general.

Now that generative AI is getting better and better, are we still going to pay these really high legal fees? Is law going to become, you know, legal representation going to become affordable and more accessible? I'm excited to actually have someone that can. give us great insights on today's episode of Everyday AI. Welcome. Thank you for joining us. My name is Jordan Wilson. I'm your host. And if you don't know, Everyday AI is a daily live stream podcast and free daily news that are helping all of us, all of us everyday people, learn and leverage generative AI. So we're going to do that. And I'm very excited to talk about today how AI will transform the business of law. I'm very excited for you all because we have a great

with just extensive background in this field. So before we get to that, as we always do, let's go over the AI news. And if you're joining this live, let us know right now, what are your questions? What are your questions? Get them in now so we can tackle them and answer them live on show. That's one thing about everyday AI. That's a little unique is we take your questions live and get them answered by experts. So, all right, let's jump into the AI news because there's actually some big.

big, big things going on today. Let's start with Samsung. They're making some historic moves. So the electronics giant just launched a generative AI model made for its own Samsung devices called Samsung Goss. The presumption is that the Samsung Goss model will come to Samsung's popular smartphones, such as the Galaxy S24 and looks like they're probably going to be Apple to the punch. And the first to bring generative AI to a major mobile device.

Samsung Goss is named after the legendary mathematician Carl Friedrich Goss. I think I'm pronouncing that, right? And includes language, code, and image functions. All right, Samsung's not the only big tech name making some moves. Amazon as well. Y'all have heard of that, right? This little company called Amazon, I think they sell books. But they're entering now the large language model game, reportedly.

So they are working on their own large language model. code named Olympus, according to a recent Routers report. So Amazon is investing millions in training a large language model called Olympus with two trillion parameters, which would make it one of the largest large language models to date. Amazon has obviously already been investing in a lot of other companies and their large language models, specifically a reported $4 billion. That's billion with a B, yes, investment in Anthropic and their large language models. language model Claude 2. All right. Last but not least, META is requiring advertisers to disclose

AI use in political ads. So a freshly released announcement from the Facebook and Instagram parent company is saying that this new policy applies to Facebook and Instagram ads and requires advertisers to disclose when AI generated or altered content is featured in political, electoral or social issue ads next year. Obviously, this decision comes as lawmakers and regulators are preparing to address the use of AI in political ads as we get closer ahead to the 2024 presidential election. My only comment on that is like, what took so long? Like we've seen, you know, leaders from both parties, you know, already use AI generated ads. So I'm wondering what took so long.

Like, we know that you shouldn't be, or it should at least be disclosed in political ads, right? All right, but you did not come here to talk and chat about the AI news. Maybe you did, but you're probably wondering about how AI is going to transform the businesses of law. I'm extremely excited about that one. And for that,

let's go ahead and bring on today's guest. I'm super excited to have Enom Hock, a consultant at Law Beta. Enon, thank you for joining the show. Appreciate it. Yeah, yeah, no problem. It's early in the morning here, but we're ready to go. Yeah, he makes the elite list of people that have joined the live show from the West Coast. So thank you for that, Enam. Maybe just let's start high level.

Give everyone just a real quick background of your career and, you know, kind of some of your experience in law. So we can kind of set the stage here. Oh, of course, of course. So I spent the first 10 years in my career at a law firm in New York, you know, doing corporate finance. I spent the next maybe eight years or so at Moody's, the debt rating agency, where I'm studying legal risks and incorporating some of that into the ratings platforms and their own analysis. And we were doing a lot of manual work, but really turning this unstructured documents into structured scores, things like that that investors could use.

So I spent almost 20 years doing that. And really just, I'm the type of lawyer that actually does not, has never been to court. you know, nothing like, you know, fancy like that, like not the ones you see on TV, more or less the negotiating lawyer, the one on the phone. But what's interesting about that experience was that I really just only did one type of law my whole career. And then I subsequently did the same thing at Moody's. So I'm really, really focused on leverage finance, leverage loans and corporate finance. And that's sort of my legal background. But I've been a technologist pretty much all my life. It's kind of my my passion, my love, I've been into it as far as I can remember, maybe 5,600 bod, comp,

you know, comp you serve internet in 1988, maybe. So it's been a while. And, you know, it's just, and now I'm getting to, now that the law and sort of artificial intelligence, machine learning and these technologies are sort of colliding, it's everything I love now all packed together. So that's kind of the skinny of it. It's, it's great. And I kind of feel the same way, right? Like, I've always, even for me personally, I've always been kind of a dork and I've always been fascinated by the creative side and the tech side, new internet innovations. And, you know, for yourself with your law background, I mean, maybe let's even talk like, how did this culmination of events and kind of your creation of law beta? How did that kind of all play out? You know, the seed for it, honestly, was probably in high school when I had done a, in 1997, I did like a,

whole presentation on a legal, a law firm that was going to be online. It's going to be like law.com, something like that. And so the seed was definitely planted there. I kind of had an idea that I was going to be a lawyer for a long time. I think what really accelerated all this for me personally, it was probably COVID, to be honest, being in New York City, sort of being trapped in an apartment for, you know, a year. I decided to really lean into that sort of area of my life that I hadn't really hadn't had time to really focus on. So it was like learning. Python, taking, you know, refreshing, taking like Harvard's free CF50 class, like doing all those kind of things and getting, getting back into it a little bit just because that was my passion.

That's what I actually would like to do. And so that, that sort of culminated, obviously, with perfect timing about, you know, these large language models and, you know, the paper from 2017 and then open AI and everything just sort of came together. So that was, that was basically it. Yeah. And just, you know, generally speaking, what is kind of the focus that you're doing with law beta? You know, is it more helping, you know, others kind of understand the kind of AI and the technical side of law and where it's going? Yeah, yeah. And on a broad level, too, I should say, because I know you've had a few speakers in the past maybe touching on law. So my particular perspective is really through the prism of sort of elite law practice in New York. you know, firms like internationally, things like that. But that's just one segment of the law, you know, that I happen to be specializing in. But I think if you look at it from all the sides, I mean, these technologies are going to improve the whole landscape. So for me, yeah,

it's definitely sort of getting, I mean, lawyers have historically not been tech savvy, like these firms and some things. And I'm not going to say all lawyers, because there's an incredible ecosystem of lawyers, at least right now, too, that are just gunning for these technologies, pushing, you know, full pedal to the metal. But, um, Historically, if you look around, the IT departments were kind of lagged behind. There wasn't a huge investment in this stuff. Machine learning tools have been around for a while on Wall Street, and they've pitched them to firms and radio agencies and things like that,

but it didn't really hold. You know, it didn't come through. And I think what's great about this time, and I think you can appreciate this too, is that I think honestly, chat GPT has just captured the imagination of people, to be able to interact with it, to use it. You're not talking about having to go train a document or a system or something like that, but just to kind of see what it is, it's just incredible. And I think that's really the spark that was needed to accelerate the adoption of these technologies,

which is my whole goal with law beta, is to try to really accelerate the adoption of these technologies. I'm eager to work with partners and with clients that really want to take advantage of these systems and not just get to the middle ground, not just get to where everybody's going to be. Everyone's going to have a large language model of some kind in their firm and their systems and their law practice, whatever. But it's almost like my brain space is always about, okay, and then where do we go, right? Then what can we do? How can I differentiate ourselves? How can we differentiate ourselves?

How can we gain a competitive advantage here? How can we start using all these data lakes? Adobe just introduced an entirely new way to create, bringing the power and precision of its creative suite into one conversational experience. Meet Firefly AI Assistant, now live in the Adobe Firefly app, the all-in-one creative AI studio. Powered by Adobe's creative agent, Firefly AI assistant lets you start with your vision, just describe what you want, and shape the outcome as it takes form with the assistant. The assistant orchestrates multi-step workflows, drawing on 60 plus pro-grade tools across

Adobe Creative Cloud apps, including Photoshop, Illustrator, Premiere, Lightroom Express, and more to help bring your ideas to life. You can also get started with creative skills, a growing library of pre-built workflows for common creative tasks like batch editing photos, creating mood boards, portrait retouching, and creating social variations. Every step the assistant takes is visible so you can refine, redirect, or take over at any time. You stay in the driver's seat as the creative director. Adobe Firefly AI assistant now in public beta.

See it today at firefly.adopi.com. Yeah. And I'm, oh, I can't wait for the rest of today's conversation. I have so many questions. And hey, as a reminder, everyone joining us live, Jay saying thanks for joining us, Enum. Thanks for joining us, Tara saying good morning from Nashville, Val. Good morning. Roderigo.

Good morning. But yeah, what questions do you have about kind of the future of law and what this means, kind of the business of law with AI? I'm going to start here. I'm going to start here. Let's go to the end. How do you think this is ultimately going to play out? Because I think, you know, I'm like, for four good reasons.

right, like probably big companies that are have big, you know, law firms on retainer are probably hesitant to use large language models. But I know that there are even, you know, specific large language models now that are being developed specifically for the legal side. So how do you think this is all going to shake out, you know, not asking you to predict the future, but how do you think this is going to shake out with large language models becoming so popular, so powerful and so niche that can really perform well in a certain vertical. Yeah, I mean, I think this is still just the beginning here, to be honest, and all the capabilities that people are dreaming of and actively building the whole ecosystem

of legal startups right now is incredible. And it's really just the beginning. But I would think that I think we're sort of overestimating what's going to be done in a year or two and underestimating what's going to be done in a decade here. And so I think that these technologies are just going to five. We can mark the calendar and talk in 10 years, but this is going to be everywhere. These technologies are so good at sort of getting rid of, and this is a theme that I think you've had with many of your guests on the podcast, but that gunky work, that stuff just feels like

where you don't have a lot of autonomy, where you don't feel like a good employee or you're not having a good career because you're just kind of stuck in this, and especially with law in the world of what I call the world of small print. You know what I mean? mean, like, if you ever go to a sports venue or something, you look on the other side of the ticket is like font and three, that's the lawyer's world right there, you know, like these little, you know, things are, you know, you get a beach ball the other day and it said like, do not use as a flotation device, you know, like, okay, so there's a story, there's a story there, obviously, but there's also like, that's, that's the world that they live in. It's very, you know,

unfortunately, I think with lawyers, they're very risk-averse and they're also always talking about risk. They're always thinking about what could come off. That's why documents have ballooned in size. I've done studies just to see like where a credit agreement started 20 years ago to today. It's just adding words, more and more words, because it's thinking of all the hypotheticals that could go wrong. And you're going to be really glad if something goes wrong that you had a lawyer there and you had all this stuff really prepared. But for the most part, some of this is not going to ever come up. It's all theoretical. And the best lawyers that I encounter and the ones that will, I think, use artificial intelligence tools and get creative and become sort of market intelligence platforms themselves,

They're the ones that don't give, you know, a client, okay, here's a hundred theoretical risk of everything that go wrong with the deal. They boil it down with their judgment, their own strategy, their own sort of experience to 10 items, 20 items, the ones that actually matter. And so I think that that's really where I think the practice is going to shift, at least from like sort of the corporate, you know, negotiation strategy, transactional angles. Sure. And in some, you know, speaking of some of that mundane work, and yes, we talk about that a lot

on the everyday AI show. But one thing, at least, you know, I'm an outsider, right? I have no clue. But, you know, when I think of lawyers, I think that there's obviously many different types, many different roles, you know, in law firms. But I think that they're spending a lot of time reading, you know, related case law. I think they're spending a lot of time analyzing it. I think they're spending a lot of time writing based on their analysis of the reading.

So when I put those things together, I say and I think can't most large language models and we've seen the testing, can't they consistently do that at the level of the best lawyers out there? I mean, is that the case? Is that not the case? Help us all understand that. I mean, I think it is the case. And that a lot of that sort of analysis about, you know, like the work I'm doing right

now is about like how can we sort of ingest these things and have the computer or the system sort of spit out that top 10 issues list, the top 20, to understand somehow the risks involved in there. And yeah, a lot of the work is that. But I will say that freeing up from that sort of road, you know, you don't want to look through 300 pages of an agreement just to really care about the 10 pages. And the lawyer spent a lot of time sort of getting there to that point. You're just going to kind of get to that point. A lot of legal documents for the most part are sort a boilerplate. You just didn't have to deal with it. A lot of it. I would say like 80% of it. The fireworks are all in that, you know, the 80-20 principle. Same thing applies in law, right?

It's the 20% of the words that actually have the most impact. And that's where imagine now, you're free to just focus on that. It's being served to you really elegantly by a system. And then you can just spend all your brain space there thinking about that, not worrying about sort of how to summarize something or extracting information or trying to find outliers or precedents or you're doing a lot of control F to find things and all that's going to be gone away. Now you'll have time to really think about, I mean, I actually think this is going to improve the practice a lot quite a bit because you're just going to have, right now everyone's kind of doing their own thing. I know there's a big movements now and I love to see this like one NDA. Like everyone's trying

to move to like let's get to like forms that we can all agree on and get rid of all these like millions of, you know, sort of firms that are forms that are out there. That's a step in the right direction for sure. But, you know, it's that kind of work about like, okay, get rid of that and let's focus on the real stuff. Like, a lawyer really should be involved. And again, I'm doing this all from like a corporate lens. But what's the company you're representing, your client? What industry are they in? What are the hot buttons there? All the research that goes into, what kind of problems have they encountered? Not just like legal is not a vacuum. You know what I mean? Richard Tromans from artificial intelligence is a good friend of mine and holds a lot of conferences. He just did this great think

piece about how just legal tech is not a vacuum. And at first, you know, I read it and I'm like, well, that doesn't seem that insightful. But if you really read his piece on LinkedIn, it's, it's profound because it just means like this legal is never just on its own, right? It's always tied to a client, to an industry, to, you know, a government regulation, whatever it is. It's so large that it'll have sort of tentacles everywhere. So you can't really just think about it in a vacuum. and lawyers will no longer be in a vacuum when all these tools are available. Yeah. And it seems like to me that there is the very infamous case, you know, a couple of months ago of the New York attorneys who submitted essentially information to the courts that contained hallucinations from chat GPT.

So they use chat GPT to help, you know, write a summary or a brief submitted it. And it turns out that they weren't using chat GPT correctly. it hallucinated it lied. Do you see that as a potential ongoing problem? And also did that incident in its high visibility in the earlier parts of kind of the Gen AI boom, did that set the legal space back? And, you know, people were too afraid. Man, I heard a rumor that somebody thought that that was a plant by the New York Times just

to slow this thing. But, you know, I will say this. First of all, I don't think, I think that's all going to be solved, right? They've already, if you look at chat GPT today, it's got sources, it's doing real-time data analytics on like actual, you know, truthful foundation matters, ground truths, things like things like that. That will all be sorted out. I will say too, because I have to give an ode to the engineers that I work with a lot because I really try to study these technologies, not to program and engineer them, but to understand the theoretical limitations of them. I need to know what they can and can't do for my clients. And one thing the engineers always tell me is like, everyone calls it hallucinations, but they call it creating.

Right? And you do want that system to be able to be creative to give you ideas. You know, people think like, oh, well, if it's trained on all this data, it's just going to know what it's already exists. But it's really not the case. And again, this is delving a little bit into, I don't think this is a hot take, but it's delving a little bit into like, well, these things are like prediction machines. They can do probabilities. They're very good at that. And they can take, you know, all that to language. But, you know, just remember that we communicate in language is the most effective way. other than in-person communication with language, probably the most effective way to communicate, but also like our thoughts themselves are just language, right? In your head, most people have a little voice that talks to them that they think is them. So there's a lot of things about what patterns and predictions it saw there to recognize that to be able to interact, you know?

And you can see like opening eyes announcements even from Monday. I think they're thinking the future of computing is not going to have one of these anymore, right? I think it's, they're really leaning towards you're just going to, you know, it's going to be like Star Trek, you know, that you'll just talk to your computer. computer and all that stuff. So it's definitely where I think things are going to head with that hallucination issue and all that. It probably was good for the space to take on a little bit of a

break because I felt like law firms particularly were starting to panic. You jump into bed with the wrong vendor. You know what I mean? You bet on the wrong horse, the wrong technology. You get dazzled by an AI, you know, demo. I always tell my clients, like, never just be like, oh, show me your AI. I want to see it. And they dazzle you with, and demos can be anything, right? Like, I've done demos before. Like, you can make them up. And the text isn't even real behind it, you know. But sometimes, you know, when you put on that show, that's not the way to go.

The best way to approach it, and I'll give this advice totally for free. You don't have to consult with me or anything. Yes. We're giving money here on the show. All right. Here we go. Here we go. Here's a $550 advice, right?

Always come with a problem, a real problem. Because the first question is whether or not AI is even the solution here. I mean, yeah, it's the sexy thing right now. But like if it's a one-off project and you just need it done, most times a manual human, that's going to be the way to go until some technology comes that maybe improves it. But it's no sense of you spending that money and R&D or CAPEX or whatever it is to get that solution if it's not really needed. So, you know, it's obviously works for repetitive things. And it's going to get to be, it's going to start impacting work that's even not just repetitive anymore. I think it'll be able to, I heard somebody talk about how.

Oh, it'll probably never be able to do like a Supreme Court brief, the way that like appellate lawyers do Supreme Court briefs. I don't think that that's the case. You know, because I could feed a machine, all every elegant Supreme Court brief going back to the 1800s, get principles extracted from them, incorporate that back in, and then add, because these systems, right, you can just add disparate knowledge and have it folded in.

So you take a book like Robert Keanu's persuasion, all the principles from that got in. You take whatever, death by PowerPoint principles. throw that in. And then all of a sudden, it's going to generate for you an argument that's got psychological components, it's got the legal components, it's got the flash, that's got everything. And so that's where I think, like, why would you say that these briefs are not going to be able to be that good? It's just a very short-sighted. Yeah. And I'm going to throw on a personal story, and then I want your take on this. So, you know, I started a venture company about nine

months ago. And so when I was looking for attorneys, one of the first things I asked them is, how do you use artificial intelligence in your practice? Do you use it? Obviously, I hired the person that I thought was using it correctly because I believe that's the future. And even when you think, okay, well, I want to pay an affordable rate, I don't care if I'm paying someone more an hour if they're using their hours correctly. How do you see even the accessibility for the everyday the average person or the average small business, the average entrepreneur, is it going to become more affordable to hire attorneys? Are we going to see kind of, you know, AI first or AI-led companies that are very high

quality and use this technology and they're going to start to create an own category? Or do you think it's still going to stay how it is for a while? What are your thoughts? I mean, I think in the short term it's going to stay a little bit how it is for a while. But I think in the long term, I think personally what excites me the most about this space. And it's not necessarily the work I'm doing right now for my clients, but I'm really, I'm really trying to barrel into that space here. And I've had great exposure to some of the leaders of that movement, you know, at Stanford or at Berkeley out here in Silicon Valley and

California, is just as equal access to justice, right? That's going to be the thing here. So it's not really the business of law, but there's a huge unmet legal need in the United States just because people can't make it work. The numbers work, right? People don't have. a lot to, everybody really does need a lawyer or has legal issues, right, a lot. You know, it could be evictions. It could be just credit card debt, identity theft, whatever it is, right? You usually need a lawyer to kind of navigate and go through that. But a lot of times there's all this unmet need because people, even people that are just like maybe not working at firms that are charging, you know, $700 an hour, but something much more reasonable. And they still can't really

cater to that clientele. This is the most promised this technology. has to actually close that gap. But there's a bunch of hurdles in the way. One of them is sort of, there's this general principle of the unauthorized practice of law in the United States, meaning that really lawyers are the only ones that should be practicing law. And it's a regulatory scheme. And it's, you know, some people think, well, maybe is that just a competitive sort of moat that they've created around lawyering, just like, you know, other kind of licenses and things like that, or is it more? But I think that's going to just have to be like really, really modified and change because is there is there a way to even I'm just wondering now is there a way to even

police that right like let's say one one lawyer starts a firm and they're tapping into artificial intelligence and maybe a large language model maybe specifically set up for legal maybe it's doing the majority of the work and it's just the lawyer signing off on that is that allowed I mean can you even govern something like that you know if if now lawyers are spending let's just say if they're tapping in and leaning into AI so hard and becoming over reliant, at what point are they still the lawyer versus maybe if they're really leaning heavily into generative AI in large language models? That's a great question.

That's a great question because I think that that's the million dollar question here, because again, these technologies are so good that at a minimum, I think the language models are always going to get you average or better because it's going to take all of all of that data, it's going to plot it somewhere in the middle, and then obviously with prompting and other knowledge bases and things like that, it can start to improve toward more of an expert level. But it's always going to start off as an average lawyer. So everyone's going to be able to get that at their fingertips. The issue is going to be ethical issues, right? So lawyers have, like, have you actually done the diligence here? If you're just rubber stamping something that you think the

machine could do, and that's it, that may be a competence issue for you, ethically speaking, especially if something goes wrong, you know, because these systems, Let's be totally clear about it. As much as I believe that they're going to just do everything that you could possibly imagine and more in terms of like sort of the future of law, right now, the accuracy is not there, right? Let's call it 80% to start, right? Which sounds good on his face, but imagine if only 80% of planes landed every day.

It'd be millions of like, you know what I mean? There'd be problems. So it's not a good number. You need that number to be 99 or 6 sigma, 99.99.99, whatever. But it needs to get improved, and it will over time. But right now, you know, if you were just to like, let's say, I'm authorized to practice law in New York, New Jersey, if I just sort of hung out a shingle on the internet and said, hey, bring it all in, and I'll just, I'll write your complaints and I'll do that.

Like, if something goes wrong, I'm going to find myself in front of the bar committee, you know, having to sort of say, like, what about what about what about doing? Is that zealous representation of my client? Am I actually doing the diligence? And that's what, you know, every bar association in the United States is having this discussion about how to fold in genitive AI. I sort of overheard a conversation two weeks ago where I listened in on the meeting of the state of California, which is a committee that's sort of addressing these issues. So it's not obviously the recommendation or it's not floated up yet.

But when they're talking about it, they were talking about like the billing. Like, okay, let's say like an average lawyer, it wouldn't take an eight hours. Now I can do it in eight minutes, right? What do I bill that as? And some people are saying, well, if you bill it at eight hours, that's fraud. And I'm like, is it really fraud if that's the market price for it? Is that going to impact hourly billing? Is this finally going to be the so-called death of the billable hour that people have been talking about for 20 years? I don't know, but there's definitely something there where we'll have to think about because these tools are going to be incorporated so well. And that's why you know that these tools are going to be huge. If all the bar issues are already doing

exploratory committees and adding rules and all that. So there's going to be an ethical landminding. And that didn't touch on like the bias of these systems either, right? For sure. And, you know, this is honestly, this has been a conversation. And I'm glad we can have you on the show because this has been one I've been wanting to have for months because I do remember when OpenAI released GPT4, which was back in March. So it's been a long time. And even at that time, their initial scores, I don't know what any of these scores mean, but apparently they took the bar exam scored about a 298 out of 400, which placed it in the 90th percentile, which I think is pretty good, but it goes to your point, if only 90 percent of

planes land, not very good. But then I guess on the other side, that's a very old model. You know, Open AI just is just released or is releasing this week, GPT4 Turbo, which is supposed to be much more powerful, a much more capable model. Like, at what point? Do we as a society say, okay, well, if generative AI, if large language models, if, and this is always like the different exams, they always have them take the exams and see, oh, how good is it? How smart is it? But at what point do we as a society say, hey, if these large language models can score in the 99th or 99.5 like percentile, like at what point do we have to shift and say, okay, what are our odds of hiring the, the lawyer that's in that point one percent? centile. Like is that is that a crazy thought to think that it might be safer to rely on this kind of technology or will lawyers just become better in the future when they're spending less time on this knowledge work and being able to think more creatively more strategically?

Yeah, I don't think the professional law is going anywhere. So no matter what like McKinsey says about the you know, I think enterprising lawyers will always find a way. And there's always going to be a need for you're never going to do bet the firm litigation and use a lab. I'll say this too about benchmarks just really quickly because I'm involved in one called Legal Bench. It's a Stanford paper. It's going to a machine learning conference this fall. And it's all about like real lawyers, real professionals like me that have been in this space

have given to computer science majors, PhDs, and sort of the tech side, all this data so that they could actually assess these models. One thing to make clear is that like sometimes, even though it looks like a model, I'm sure GBT4 they took all the precautions necessary. But as it swallows all that data, you never know if it swallowed an old exam, right? And just understood, just knew the answer because deep down somewhere in that neural network, it pulled it out, right? So there's always like benchmarking is really, really tricky, you know, because you always

have to have something that's totally segregated from the internet or not available yet to do a real time. What they do is hand score these things, like open AI for all its technology or prowess and things like that when they really get down to it, their hand score. They're grading these things by hand because that's like the old, school way to make sure that this isn't, it's a new exam and all that. So I'll say that first. But second, that other question, I think about like, you know, I don't know, I wouldn't give much credence to like LSAT performance or bar exam performance translating into law at all. And I think any

real lawyer would never would never say, yep, yep, every lawyer that got in the 99 percentile is an amazing Supreme Court litigator and a big law firm partner and all that. It's just not the case. Those things are so abstract and really not related to law at all. You know, maybe there's some sort of logical thinking that they're analyzing, but I wouldn't give it any credence. So again, it's going to lean into the future of I think lawyers are just going to be better with these tools. They're going to be able to express their talent in different ways to people and really start exercising strategic judgment. And I think going back to sort of what a lawyer used to be, you know what I mean, which is just, it's not a paper pusher. Just remember,

we live in a world where legal is really just looked on as a friction, like in the work that I did, they never really respected lawyers in the sense of like it was just a thing in the middle of the other thing. They wanted to do an M&A deal. And so the financing lawyers had to come to get the money to buy the company. That's not their primary objective. They just want a check, you know, at the end of the day, all that paperwork, all that legal work, just so they can send a wire. Because really what they want is to buy a business. So again, everyone's looking at it like friction. That's why I think that if it's not happening yet, just wait, but corporate clients of major law firms are going to start, especially if they're using AI in their own legal departments, you know,

companies like Ironclad, they're doing great things with contract management and all. that life cycles once clients start demanding hey we're using this we're having success you better start showing us this on your end you know what i mean i don't want to see bills inflated with insosiate hours and all that so it's going to be the practice of law in that sense of like being trained initially by your on your client's dime that'll have to change so it's going to have to be other other methods and that i go into my clients because we're building tools to even address that oh this has been a good one uh you know thank you so much so as as we wrap up here because we've taking this all over the place. And I love that you've been able to give us fact-based insights,

the fact that, you know, and we're going to share about the work that you're working on with legal bench in the newsletter today. But, you know, just as we wrap everything up and as we look forward, what's the one takeaway that everyday people can, can, you know, use this conversation, whether they're hiring an attorney for themselves, maybe, you know, someone that's going into the legal field, you know, maybe they just passed the bar, you know, what's, I mean, what should people be be looking at and paying attention to to help make sense of generative AI in the future of the business of law? I think for future lawyers, you know, the people that just taking the bar exam or whatever know that these tools are coming. And so if you're not already conversant in them,

get conversion in them because it is going to dominate the industry in 10 years. That being said, you don't have to necessarily don't, I think, and this could go broader than just lawyers, But no one should feel like they missed the boat here that these tools are just off and done. And it's all, you know, like these large language models could only be created now. Like we're living in the now and this is when they were available because of all the data and all the compute that was available. It just took this amount of time. So everyone's here together and we're at the ground floor. And I think knowing how these technologies work is helpful.

I don't think you need to say you have to program or engineering them. but getting some framework that you just kind of roughly understand how they work is a good thing. And obviously using these tools, you know, I feel like the lawyers are going to be able to write emails quickly or, you know, what I do these days, it's incredible. Like I just take notes just kind of randomly, really, but it's so good at organizing them, packing them nice, making them more, you know, presentation worthy. So it's things like that that start using these tools right away, you know, whether it's, I would obviously recommend chat GPT, the premium version, too, do not get stuck in some. other model. But, you know, look around. There's perplexity. There's other tools, too. There's mid-jurney for art. There's so many incredible things that you should be leaning on. It's not just like, don't think about it is this a work thing. This work thing can become a lot of different things.

You know what I mean? So good. So many good insights. You know, thank you so much for joining the everyday AI show. We really appreciate it. Thanks, thanks for your time. Thank you very much. All right. And as a reminder, there was a lot here. We impact so much. I'm super excited. Like people don't realize. I'm still like I'm a human. After I get off this conversation, I'm going to get up, re-listen to it, type in all the insights. So we talked about Legal Bench. A lot of other great insights dropped in today's episode. So make sure you go to your everyday AI.com. Sign it for the free daily newsletter. If you're listening on the podcast, checking the show notes. And we hope to see you back tomorrow and every day for more everyday AI.

Thanks, y'all. Meet Firefly AI Assistant. Now live in Adobe Firefly, the Allman One Creative AI Studio. Just describe what you want to create in your own words and the assistant handles the rest, orchestrating multi-step workflows across Adobe Creative Cloud apps, including Photoshop, Premiere Express, and more in one conversational interface. You direct the outcome while the assistant accelerates execution. Stand control with the ability to step in and refine at any time.

See it today at firefly.adobie.com. And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going. For a little more AI magic, visit Your EverydayAI.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.