No Priors: Artificial Intelligence | Technology | Startups - The AI Tutor For Every Child and the Next Frontier of Education, From Khan Academy’s Creator Sal Khan

Episode Date: August 17, 2023

The future of education is right at your children’s fingertips. Sal Khan, CEO and Founder of Khan Academy, joins Sarah Guo and Elad Gil this week on No Priors. For over a decade, Sal Khan has been trying to reform education, beginning with tutoring his cousins in math.  He's the father of the YouTube "chalk talk" format, and has now served tens of millions of students through Khan Academy.  He guides us through how Khan Academy is using AI to personalize a student's educational experience, transporting students into immersive learning experiences that allow them to debate historical figures, to assisting teachers with lesson plans that address the learning gaps keeping students from reaching their full potential, to a Khanmigo, a tutor for every child.  Prior to founding Khan Academy, Sal worked as a hedge fund analyst. He holds an MS in business from Harvard University, as well as an MS in Engineering and a BS in Computer Science from MIT. Show Links:  Khan Academy - CEO & Founder - Khan Academy | LinkedIn   Khan Academy Sign up for new podcasts every week. Email feedback to [e-mail] Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @salkhanacademy Show Notes:  [0:00:06] - Sal Khan's Journey [0:08:41] - Mastery Learning and AI in Education [0:19:53] - Future of AI Tutors in Education [0:23:10] - Education's Future With Generative AI [0:29:35] - Connecting Learning Through Tutoring and Collaboration [0:33:22] - Implications of GPT 4 on Education [0:40:42] - Future of Education and Job Skills [0:46:47] - Importance of Traditional Skills in Education

Saul Kahn democratized learning with his educational YouTube videos that turned into Khan Academy, which has 150 million learners from all over the world today. 15 years later, he's as energized as ever about making learning personalized with artificial intelligence. This week on the podcast, Elad and I talk with Saul about the impact of AI and education. He says, surprisingly, we're on the cusp of the biggest positive transformation that education has ever seen. Khan Academy has recently created ConMigo, a chatbot tutor that can nudge learners in the right direction if they get stuck. Saul, welcome to No Pryors. Thanks for having me.

Sal, can you start by giving us some background on yourself and how you ended up starting Khan Academy? Sure. You go way back. My original background was in tech. I go to business school. I end up being an analyst at a small hedge fund. And it was shortly into that, it was 2004. I was a year out of business school.

I had just gotten married. I was based in Boston at the time. I was born and raised in New Orleans. My family was visiting me up in Boston after the wedding, and it just came out of conversation. My 12-year-old cousin Nadia was having trouble with school, math in particular. She was being placed into a slower math track. Her parents, my aunt and uncle, hadn't gone to school in this country,

so I don't think they understood the implications that on that track, she wouldn't end up taking calculus in high school, et cetera, et cetera. So I took it pretty seriously. I said, hey, I'm up for tutoring you, Nadia, if you're up for receiving it. And she agreed. So she goes back to New Orleans. I start tutoring her remotely. She actually gets caught up with her class.

She was actually having trouble with unit conversion. She gets a little head of her class. I call up her school. I said, I really think Nadia Ramon should be able to retake that placement exam. They say, who are you? I say, I'm her cousin. And they led her. And that same Nadia that started off as, for lack of the word, a remedial student,

was then put into an advanced math track. So I was hooked. It was fun to stay connected with family. I really enjoyed geeking out on the math. And it seemed to be really helping my family. And so I started tutoring her younger brothers, word spreads in my family free tutoring is going on. Before I know it, I'm tutoring 10, 15 cousins. And I still had the day job.

I was working as an analyst at a hedge fund. And I saw a common pattern with my cousins. They just, the main reason they were struggling was they had gops in their knowledge. Oftentimes, the reason they were having trouble with that algebra equation was because they weren't fluent in dividing decimals or negative numbers or exponents. and so I started making software for them, which I did for fun because I, that part of my brain wasn't being used fully in my hedge fund job. I started making software that would generate problems for them, give them immediate feedback, provide hints if they needed them, allowed me to keep track. I put a little database behind it and allowed me to keep track of what they were doing and when

they were doing it and what they were getting wrong and right. And I called that Khan Academy. That was the domain name. It was available. It was kind of a fun family project. And about a year later now, we're about 2006, and I was showing this off at a dinner party. And by this point, my family had moved out here to Silicon Valley. Did you still have the hedge fund job? The same hedge fund job. But actually, it's interesting. I have a whole theory that we can keep referring to, that benevolent aliens are using Khan Academy to prepare humanity for first contact. And because there's a series of things in my journey that seemed like it was almost like, you know, people were moving me around in a way that that benefited this. But one of the

of them was my boss. I mean, it was a two-person hedge fund at the time. His wife decided to become a law professor. And it was interesting because at first she thought she was going to go to UVA. And in case, you know, we would have ended up living on a farm in Charlottesville or something. But at the last minute, she decided that she wanted to take the job at Stanford. And so my wife and I came out here and my wife, who's a physician, she ended up doing her fellowship at Stanford because of my boss's wife, not wife's choice. But that landed us a here in Silicon Valley, which I think ended up becoming very important for the future of Khan Academy. And then I was showing this off at a dinner party in Silicon Valley to a friend, and he said,

well, you know, this is cool, but how are you scaling your lessons? I'm like, well, I'm not. Like, it's hard to do with 15 cousins what I was doing with one. He said, you know, there's this YouTube thing. Why don't you record some your lessons as that? And I immediately thought that was a horrible idea. I thought YouTube was somewhat frivolous for dogs on skateboards, cats playing piano, et cetera. And, you know, I was like, Yeah, how do I even record that? This was before phones had cameras, et cetera, but I decided, oh, there's got to be a way to record your screen while you're drawing something,

and it's called Screen and Capture. And I just started using it, and I started just covering things that I found myself repeating a lot for my cousins. And my cousins, famously or infamously, told me that they liked me better on YouTube than in person. I think what they were saying is they liked having an on-demand version. There's no judgment, no shame. if they had to review things, it was available all the time. But they really still appreciated having me in their life, I think.

But then, you know, YouTube, just a lot of people started discovering it. A lot of people were also discovering the web app. I actually had to shut down registrations because it was crashing my $30 a month, Java Servlet web hosting that I was using at the time. At what point did you open it up to anybody that wasn't a cousin? Well, YouTube videos by definitely, you know, I saw no reason to make them private. Right. I meant the Java Servlet app. The Java Servlet app, initially I had opened up to anyone who wanted to register.

And I was actually, you know, my day job, I was an analyst at a hedge fund, but I was emailing local schools and seeing if they wanted to use it. And some of them were interested in using it. Famously, the first school that used Khan Academy, this was in 2007, was Sidwell Friends, which is, I didn't even know. They're very elite school in D.C. It's where all of the, you know, senators and presidents kids go. But they were actually a fifth grade classroom, This teacher was John Mormino was one of the first, and he was just using this home brew software that I was creating. But I actually stopped registrations on that at 10,000 because, once again, it was crashing my $30 a month web hosting, and my cousins weren't able to use it. I mean, talk about backwards Silicon, you know, so many things in Silicon Valley, people would love to have the problem of an app that more people want to use.

I was like, nope, no more for anyone else. But by 2008, there were about 50,000 folks who were using the YouTube videos on, on a monthly basis, which, you know, compared to where we are now, seems minuscule, but at the time seemed like a lot compared to my 15 cousins. And at the time, I didn't think I was going to quit my day job. I actually liked it. I was getting paid well. But I did set up as a not-for-profit thinking that maybe I could get some funding,

maybe do this on the side, hire some folks, et cetera, et cetera. So I set it up as a not-for-profit mission, free world-class education. But then a year, fast forward, it kind of took over my life. At that point, it's like, you know, 100,000 or something folks were using it. This is all I was thinking about. And, you know, I kind of came to terms with, we were trying to save up money for a down payment on a house in Silicon Valley, which we all know is not a joke, even back in 2009. But at the end of the day, you know, real wealth is being able to do what you feel like is your purpose in life, I think. And as long as you have enough money to, you know, put food on the table, go to a restaurant every now and then, hopefully send your kids to college, et cetera, et cetera.

if you can do what you care about and feel a sense of purpose, that's probably a bigger deal. So that's what I told myself and my wife, and I took the plunge to do it full time. It's an amazing story. At some point, did you, like, learn more about education? Because this feels very much like, oh, I know algebra and have some instincts about what these kids don't yet know about algebra. Like, was there, like, reading about, you know, the two sigma problems somewhere in there, like pedagogy, any of it? Simplencers, in the early stages, no. In the early stages, it was, very intuitive driven. And it was really, you know, I think a lot of, I'm guessing that both of y'all,

we were all in the category of kids that did pretty well academically when we were in school. And I think when you do well academically in school, there's one of two narratives you can tell yourself. One narrative is, hey, I'm just special. I'm just good at this. I'm just gifted. The other narrative is, you know, what I'm doing, it doesn't really feel all that special. And some of my peers that are flunking the same classes that I'm getting A's in, they can beat me at chess. They can, you know, learn a video game or a puzzle or solve a puzzle faster than me.

So I don't think it's about like pure cognitive capability. And then the more, I used to tutor peers in high school as well. And the more I did that, I just realized it was the same thing I saw with my cousins. And the benefit I had, and a lot of us that did well in topics like math and science, we didn't allow those gaps to fill whenever we felt a little bit uneasy about how fluent we were with something. We gave ourselves a little bit of extra practice. And we weren't just memorizing things. We always wanted to understand why, you know, why does the long division algorithm work?

You know, let me think about that. Does it, okay, that makes intuitive sense. Why am the place values work, et cetera, et cetera. So I think when you do that, then later and later, math, frankly, gets easier and easier, not harder and harder. And if you do the opposite, where you don't understand the conceptual ideas behind the algorithms and the equations, and then you also have fluency gaps. I mean, the number of classrooms I now visit where seventh graders, they understand exponents,

but they literally, you know, three times seven, they're grabbing their calculator. I'm not exaggerating. I literally saw a seventh grader in the Bronx for three times seven grabbing their calculator, and I literally slapped their hand. I don't know if that can get in trouble for that anymore, but like I literally slapped their hand.

I was like, no, you figure out three times seven, and the kid, he started to use his fingers. And I'm like, this is on some level malpractice that no one sat down with this kid for like an evening and said, you have got, like, it has to be automatic. It's got to be in milliseconds that you know what three times seven is. because he understood all the concepts of exponents and algebra and all of that, but the cognitive load of having to do that.

So anyway, I was experiencing that from a very intuitive point of view. Obviously, once Khan Academy got more traction, started working with more educators, people would tell me, oh, you're just talking about mastery learning. I was like, yes, that's exactly what I'm talking about, which is just that you always have the opportunity and incentive to fill in your gaps. Oh, you're talking about differentiation, that everyone is learning at different paces. And I was like, that's exactly what I'm talking about. And then people are talking about, well, yeah, mastery learning, Benjamin Bloom,

even though it's arguably the oldest way of learning that you keep working on something, if you haven't mastered it yet. He coined it as mastery learning. He also wrote this whole, you know, Two Sigma problem, that if someone gets one-on-one tutoring, personalization, mastery learning, can get accelerated dramatically. So, yes, it was intuition for me that seemed to work, but now I'm very familiar with a large body of literature

that actually backs it up from a pretty, pretty rigorous. Could you talk a little bit about the range of things that Khan Academy does now because you have everything from Conn Labs to Camigo to a variety of other efforts? It'd be great to just kind of get a view of, you know, broadly what you focus on currently and what are the areas you're most excited about. Yeah, and the answer I'm going to give you will probably make any like textbook strategy person cringe. Because, you know, oftentimes people say, just focus on one thing. And maybe it's coming from me that I feel like the opportunity to transform education is not. going to happen if you just do it narrowly. So I view our charter, you know, the mission free world class education for anyone anywhere, which by itself is a very big statement and almost delusional when I was one guy operating out of a closet, you know, a little over a decade ago. But we are, we are covering all of the core academic material from pre-K through the core of college. That's the goal. We want to do it so it's personalized. We actually even want to do it so it can eventually result in credentials. And so what we are doing already on Khan Academy, so we have Khan Academy, so we have

Khan Academy Kids, which is used by nearly 2 million kids, primarily in America, it's actually kind of turning into like the Sesame Street of this generation. Really great efficacy studies. Kids are spending an average of 90 minutes a month on it, which is pretty high for an average. That's a large chunk of all of the kids in America between the ages of 3 and 7 who are using that. That's all subjects. That's math, reading, writing, character development, social, emotional, whatever you want to call it. Then as you get into older grade levels, it's called Big Khan. That's where we just keep going from just the basics of math all the way through, calculus and statistics and multivariable and on and on. Science, we have science, a pretty

strong progression from, I would say, late elementary school through early college as well, going all the way to biology, chemistry, physics. And this is not just videos. This is, most of our resources are actually behind the exercise platform, where you can get as much practice as you need, deep item banks, immediate feedback. Teachers can keep track of what's going on and assign through the platform. We've added humanities, so, you know, American, history, civics and government, world history. We have art history. So we're filling out that grid of all of the core academic material, financial literacy, computer science. So all of that economics is happening. Early on, in 2012, 2011, I wrote a book, The One World Schoolhouse, and the first third of the

book was kind of like the history of education. The middle third of the book was my journey getting here. The last third of the book was, in a world where things are changing and tools like Khan Academy exist? What should education look like? It's almost like a first principles exercise. Could we have full year schooling, mixed age classrooms, personalization mastery throughout, more time for hands-on if you can do the other stuff more efficiently? One thing to write about it, a whole other thing to implement it. On top of that, my oldest child was about to enter kindergarten, and I felt like a hypocrite if I was telling everyone to do mastery learning, and if my kid just went to the local school that was not doing mastery learning. So we started con lab school, literally

underneath on the first floor of the offices of Khan Academy. That was back in 2014. That is now a K through 12 program. We've had three graduating classes already, so I'm happy to talk more about that. We started another online school with Arizona State called Khan World School. We just started last year with that, but that's now sixth through 12th grade, and there's some very exciting things happening there. And then most, well, I'm glazing over a lot. We are working on credentials, So we have a pilot with Howard University where kids in Title I high schools are getting mastery on Khan Academy,

and that's resulting in college algebra credit, which solves a major need, happy to talk more about that. And then most recently, we're doing a lot with generative AI. We started partnering with OpenAI about a year ago. So well before, it was really a big thing, and we had to keep it quiet for a while.

But Conmigo now, you know, it's in our minds, it's a tutor for every student. It's a teaching assistant for every teacher. It can support students while they do traditional work on Khan Academy, answering questions, making it relevant, motivating them, even advising them. We have functionality where it can be a guidance counselor, it can be an academic coach, but they can also debate the AI and practice their fine-tune their argumentative skills. They can talk to simulations of historical or fictional characters, and on teachers, they can use it to create lesson plans, grade, create rubrics, refresh their own knowledge. You've said that we're on the cusp of the biggest positive transformation that education has ever seen. How important do you think AI is relative to the broader set of access that you have through things to YouTube and online coursework and all the rest of it?

Is this a complete game changer? Is it an add-on? What's the relative degree of importance of this shift right now? I think in the very short term, it is going to be a meaningful add-on. I think if you go three to five years in the future, it will be a game changer. And the reason I say that is going back to Benjamin Bloom, but I think this even predates Ben, the goal standard was always to have a personal tutor. You go back to, if you were a prince in most of, if you were Alexander the Great 2,300 years ago, you had Aristotle as your personal tutor.

And Aristotle would speed up, slow down, motivate you when you're feeling down, like do all of these things with you. And that was always the goal standard. Two, 300 years ago, utopian idea of mass public education, but in order to do that economically, we had to make compromises,

one of which is you don't get a personal tutor. don't get a one-on-one teacher, we're going to batch you into groups of 30, we're going to move you at a set time or pace, we're going to apply some lectures and standards and homework, etc. On the test, some of you're going to get 100%, some of you're going to get 80, some of you're going to flunk it. Too bad, the batch needs to move forward, and somehow we expect those of you who didn't know 20 or 30 or 40% of the material on the simpler stuff to understand the more advanced stuff. And what happens, those gaps accumulate and kids start falling off. And then you eventually put kids on different assembly lines when you start tracking them.

And that kind of worked during the Industrial Revolution, where you didn't need a lot of people in the knowledge economy. You needed kind of basically educated people to work in factories, et cetera, et cetera. You know, not so acceptable anymore. But that was a compromise that we had to make. Benjamin Bloom, 1984, comes up with a, writes this two-sigma problem where he showed one-on-one tutoring, two standard deviation improvement. Two standard deviations takes someone from the 50th percentile to the 96 percentile.

So an average student then becomes a very strong, becomes an exceptional student. He calls it a two-sigma problem because there's no way you're going to be able to do that in a real classroom. Like we don't have the resources to give everyone a tutor. And then he even, back in 1984, tries to see, well, what could you potentially do if you approximate a tutor using technology? And he kind of throws out the conjecture that you could get one standard deviation of improvement. And to the large degree, everything that we've been doing at Khan Academy was how can we approximate, start to scale pieces of what a tutor could do. So a micro-explanation, that's what a video does, and it's on-demand whenever you want it. Exercises with immediate feedback, that also starts to

approximate. And then give teachers and give tutors and give parents information so that they can also act more like tutors, as opposed to just giving a lecture to everybody. They can look at the data where kids are and then do more focus interventions, have groups of three or four, the kids that need help with the negative numbers while the other kids keep working. Now work with the five kids who need help with decimals while the others. So it was all about, okay, in a class of 30, can we help approximate tutoring? Well, generative AI, and I didn't think this was going to happen in my lifetime. So when we started playing with GPT4 about a year ago, it blew my mind that you could actually get it not only to pretend to be a tutor, it actually had good, quote,

tutor moves. It was being truly Socratic. It had some big issues with it around the math and the hallucinations. We've been working pretty feverishly to mitigate those pretty significantly. It's pretty darn good now. And so it's already, and we've already started putting it out in real schools out there, we're already seeing its increasing engagement. More kids are not only engaged, but they're not getting as blocked. Teachers are definitely, you know, we haven't done rigorous studies on it yet. It's very early. But anecdotal, teachers like, that's answering questions that the students either were afraid to ask or that I as a teacher would not have been able to get to. Teachers are already really enjoying the teaching assistant functionality. It's saving them a ton

of time, writing lesson plans, rubrics, they kind of feel less alone as a teacher. And now we're looking at ways that it can even help address some of the problems that generative AI introduced, where we actually just made an announcement today with instructor, the folks who make Canvas, the learning management system that goes out to its most of higher ed and a large chunk of K-12, we're like, well, if you use the AI to create rubrics, lesson plans, and then the AI essentially administers the assignment with the student and doesn't do it for them, but does it alongside them, then the AI can report back not just on the outcome of the assignment, but actually the process. Like, hey, yeah, I worked with a student. We brainstormed thesis statements, and then they were

having a little bit of trouble backing it up, but that we eventually got there, and it took us about four hours. And so you can be pretty confident in that situation that the kid didn't just copy and paste from chat GPT. So I think in the next year or two, it's going to be a really useful tool that's going to increase teacher productivity and support a lot of students. I think when you start going about three years out, we're already working on the notion of memory for the tutor. So the tutor, if you tell it, if you ask the AI, why should I learn this? And it says, well, what do you care about it? And you say, well, I love women's soccer. The World Cup's going on. So that's fun of mind. So I love women's soccer. It'll say, okay, it'll make a connection to that.

But the next time, it should remember that you like women's soccer so that it doesn't have to ask you again. And so it's creating, but it could, you know, you like language in a certain, you like more casual language or more formal language or your reading level, whatever it might be. So we're actually already prototyping that it remembers these things about you. We're making that very transparent to the user so that they can say, no, I'm not into women's soccer anymore. I'm into gymnastics or whatever it might be. But we think that type of memory and the type of memory where it can refer to previous

conversations will allow it to be more, allowed to graduate from being just a on-demand help for certain tasks, to being something that could have a long journey with you. have a narrative. I know it sounds a little bit wild, but I could imagine in five or ten years you're going to get your best college recommendation from Conmigo. Because Conmigo's like,

I've been working with this kid for hours for the last 12 years, and let me tell you about this. And by the way, they love gymnastics. And you have a great gymnastics program at whatever. So I think that's, and you're going to have,

you know, we're already playing with the text to speech. I know people have heard text to speech before. What's about to come will blow your mind. It's hard to differentiate from a real human being, which is scary on other dimensions of life, but it's good for tutors. And then I think in three to five years, yeah, you're going to be able to video conference with your tutor, which then makes it like a real thing that you can have a long journey with.

Have you ever read the Neil Stevenson book, The Diamond Age? Not only have I read it. I used to give it away to people. Every employee at Khan Academy used to get the Diamond Age. And, you know, you're referring to it because, you know, it takes place in this, like, neo-Victorian not too far off future. in, like, China, and this member of this, like, Neo-Nobility gets this AI tablet app for his granddaughter to educate her, the Young Ladies Illustrated Primer, and it gets bootlegged, and it gets in the hands of 200,000 orphan girls who live in barges, and then they essentially just take over.

And so I've always used Young Ladies' Illustrated Primer to our team at Khan Academy. He's like, this is what we're hoping to build, in the long run. And I never thought we were going to fully build it. I thought we were going to be able to approximate it. But already, Conmigo can do some things that are maybe even beyond what the young ladies illustrated Primer did, where you can talk to Don Quixote in any language that you want. You can get into debates with it, et cetera, et cetera.

And I think in the next three to five years, almost everything that Neil Stevenson imagined, I think he wrote the book in 1994, I think. is actually going to be a reality, which I didn't think was going to happen in my lifetime. How does that impact the structure of the actual school? So say that you think ahead five or ten years and you're in the context of an advanced K-12, maybe it's con labs, maybe it's something else. What happens or what changes or how do you interact with the tutor versus the classroom environment?

And I know it's hard to predict the future, but I'm sure you've thought a lot about this stuff. I think big picture at a place like con lab school where, you know, even before generative AI, and at Khan World School, actually, we just documented. We were almost embarrassed to say the results, Steve Levitt of Freakonomics fame. He actually looked at it himself because he was like, is this real? And he's like, oh, it's real. I mean, these kids are learning about three to four times faster, not three percent faster or 30 percent faster, three to four times faster. Wow.

And at Con Lab School, we've consistently seen at, you know, at least 1.5 to two grade levels in a year. And this is before generative AI. So I think in some of these settings, we haven't had. the same issues that the, you know, in the traditional school system, kids are getting about point seven grade levels per year. Now, I think if you go to more fluent areas, areas where parents are college educated and they can afford more supports for their kids, you're probably seeing at least a grade level or maybe closer to what we're seeing at Con Lab School or like 1.5, two grade levels a year. But I think in environments where basic fluency has been less

of an issue, you're now going to free up more time for teachers. So they're going to have spend less time doing things like grading and lesson. Most teachers spend about half their time doing that kind of stuff. So that gives them more energy for themselves, but also more energy to just be with the students, which I think is really powerful. Students are going to feel a lot more supported. And I think if you fast forward three to five years, kids are going to feel very supported in that kind of that basic fluency and that core academic. But I think it's going to be like the magic school bus. Like you're going to be able to put on your like, Apple, visors, or whatever, that, you know,

and you're going to be able to, like, go on with the teacher and, like, you know, jump into the circulatory system or time machine yourself to ancient Greece and have a debate with Socrates. And you're going to be, like, I think you're going to have, and you're going to have, you know, one of the things we do at Con Lab School is just try to create more time and space for students' passions.

And so you're going to have a world where a middle school student is going to be able to create, like, studio quality techno music, or, you know, a Lord of the Rings quality, you know, epic, epic science fiction movie or fantasy movie. So I actually think it's going to be very exciting. And I think if you go into, let's call it, more mainstream classrooms, I think we're going to go a lot further towards solving just the core fluency issue and not just in math, but in writing as well, because that's the other thing.

The main problem with writing, it's very resource intensive. It's hard to get feedback. It's very expensive for the teacher to give feedback for 40 kids. It's mind-numbing, frankly. Kids have to wait a week to get that feedback. then they may or may not get a chance to iterate on it. And so most people aren't getting much practice. And that's why you're seeing a big problem with writing

on top of the problem with math. So I feel confident, I mean, already this year in Newark, New Jersey, which is not a special case, school district, the North Ward just started using Khan Academy pretty intensively in November. This is pre-generative AI. As of June, 70% of those 6,000 kids use Khan Academy at a level that's associated with pretty profound

efficacy gains. We're still waiting on those test scores, but I think we're going to show for the first time in, like, U.S. history or recent history, a pretty large acceleration of a large urban school district in math. And that's pregenerative AI. So I'm feeling actually very optimistic about that. And then, yes, especially for those students whose families might not get an opportunity to, you know, put them in AI summer camps like we do with our kids or, you know, take trips or, you know, I think AI is going to bring the world to them in really powerful ways. They're going to be able to practice their debating skills. They're going to be able to connect things to other ideas.

They're going to be able to create pretty profound things. They're going to be able to also go on that magic school bus. And it's not going to be something that requires a ton of resources. So I'm overall hopeful. There's always forces of friction and cynicism and et cetera, et cetera. But because teachers have a lot to gain here, it's going to save them a lot of time and energy. I think you're going to see a faster uptake of this. than you see of other solutions.

There's the old saying that the future is here is just not equally distributed. And with things where you have sort of strong structure in place, like the education system, is this a five-year transformation, a 20-year transformation, like how long will it take for schools to adopt this

despite all the incentives to do so? Yeah, I've learned not to be too aggressive in my predictions, but I think it will be safe to say that in five years, almost all teachers will be using generative AI, for lesson planning, grading, progress reports, rubric creation, I think that is already,

that might happen by late next year, honestly. So that's going to happen fast. So I think in a world where almost all teachers are already using the tool, I think having a generative AI tutor supporting the student,

I think that's for sure going to be mainstream in three to five years. I think, you know, because ChatGPT introduced the emergency around cheating, Like, it's not like it's a nice to have. It's like a problem. It's like broke the education system.

People need a solution. And so I think that has to be fixed in the, and I think it's actually going to be fixed with generative AI as well. That has to come into play in the next year or two. So, you know, five years in the future, when you just visit a classroom, you might not superficially see a lot of differences. But I think when you start double clicking on it and you realize that,

wow, teachers have a lot more time now for student-facing things. A lot fewer students are stuck. there's a lot more space for differentiation now. I think that will be transformative, and I think you're going to see it in student engagement, motivation, and in things like test scores. So in this novel, The Diamond Age, one of the things that happens is because these tutors are so powerful

and they have memory, right, some of the things you're working on, and they're engaging because they have narratives and the dragons that's like teaching you calculus and physics with your environment, there's a lot of engagement with that and much less interest in some cases in sort of like group schooling settings?

Like, can you talk a little bit about how you think about that? Because you also started Schoolhouse, Schoolhouse.World, with our mutual friend, Shishir, founder of Koda, and, you know, this is like small groups online and believing that there's still the importance of this. Can you talk about sort of how those pieces fit together and what the motivation was there? Yeah, absolutely.

And, you know, just a little bit more detail in Schoolhouse. We started during the pandemic. You know, that was a moment where, like, Khan Academy's usage went up three acts. And it was clear that it's great if people are using Khan Academy, but they needed, everyone was isolated and they needed more human support as well. And I'm always thinking about, okay, how can you give something at scale that used to feel really expensive? And tutoring, live human tutoring, for the most part, is expensive. But I was a well, what if you could leverage volunteerism? And so it was a bit of a, you know, optimistic idea that there would probably be a lot of people who want to do, you know, what I did with Nadia, that a lot of people would actually love to do that.

And so we set up schoolhouse.world as a way, you know, it's another nonprofit, a sister nonprofit to Khan Academy. Its mission is to connect the world through learning. And the whole idea is, is that you get free tutoring and who gives the tutoring, it's volunteers. And a lot of those volunteers could be near peers. They could be high school students, college students, but some of them also are teachers who teach at a fancy private school and want to give back. They might be a retired professor, et cetera, et cetera. It's gone well. I mean, it's definitely at a smaller scale than Khan Academy. It's on the order of schoolhouse is on the order of about 10,000 folks per month, but it's doing some pretty powerful things. Already a large fraction of the kids who

take any given SAT are already using schoolhouse to get live human tutoring. Khan Academy had been the official, has been the official practice for kind of the asynchronous tutoring aspect of the SAT or the individual practice, but now you can get, you can join cohorts and become part of that. And, you know, I think the power of schoolhouse is as powerful as, is to get help academically, I think that's social connection, especially social connection across borders, et cetera, et cetera, hugely powerful. Same thing with con lab school, con world school, we think it's all about the human to human connection.

Con world school, which is an online high school, but its anchor is a Socratic seminar where kids are getting together almost every day and they're debating things. Now, it's not five hours on Zoom. We think that's wrong. That's mind-numbing. That was what was wrong with pandemic schooling. Like, that's literally going to ruin kids' eyes and, like, ruin the. them. But you know, an hour, two hours, but you also don't want completely asynchronous. You want

some interaction. And at Khan Lab School, even though people think it's Khan Academy, it's downstairs from that. Of course, yes, we do use Khan Academy. But if anyone visits, you're going to see far more collaboration and interaction amongst these students and far more talking amongst the students than you will see pretty much at any other school. And I become a believer more and more and yes, it is important to improve test scores, et cetera, et cetera, and we are doing that. But that's social, emotional, that character development, that ability to collaborate with other folks, that's arguably the more important thing. And if I, I almost didn't do it by design out the gate, but if I'm one of the things I'm proudest about it, at all of these projects we're doing,

is that element, is that the collaborative element, the connected element. You know, whenever we bring a new faculty to Con Lab School, they're always like, it's weird how collaborative these kids are. Even when you get into high school, even in Silicon Valley in high school, where a lot of other schools get quite competitive, secretly quite competitive, but everyone knows it. That's not happening at KLS. The kids are genuinely collaborative. I think part of that is more time and space to pursue your passions.

I think a lot of part of it is mastery learning. It isn't about, you know, elbowing out your peer. It's about, hey, everyone here should learn it. And I think a lot of it's about the peer support, which we're trying to scale out schoolhouse, but has always been part of Conlap School. How did the relationship with OpenAI and GPT4 start and the origins of Conmigo.

Was that something you came up with and you went to Open AI? Did they come to you? I'm a little bit curious about the origins of both the product and the relationship. Yeah, you know, this story is going to be just another one of the data points behind the benevolent alien theory.

It was about a year ago, end of summer of 2022. I got an email from Greg Brockman and Sam Altman, who I was familiar with. I didn't know super well, saying, hey, we're working on our newest model. we'd love to talk to you about it. And I was skeptical whether it would have any real strategic bearing

on what we do at Khan Academy, but just as someone who's followed the industry and has always been fascinated by this, I knew I had been following GPT2, GPT3, Dolly. So I was like, oh, are they talking about GPT4? That's kind of cool if I get to be on the earlier, get to see what's happening there. So we had that meeting,

and they hadn't even finished the first training run of GPT4, but they said, you know, we think it's going to be done in about two weeks. And we think this is going to be the model that really wakes up people

to the power of generative AI. We want, two reasons why they were talking to us. The first was we want to launch with some social positive use cases. To us, the most obvious ones are education and health care. And y'all are the first call we're making. And we want to do it with organizations that people trust and are going to, like,

do it in a thoughtful way. And so as far as I know, were literally the first people they called. And I said, well, you know, I'm up for looking at it. And the second reason why they want to chat with us is it turns out that Bill Gates, when he saw GPT3, he's like, oh, this is cool. But we all know, GPT3 really didn't have a good handle on knowledge. And he told the OpenAI team, he's like, I'll be impressed if this could pass the AP biology exam. He literally said that. And so I think part of the reason they also reached out to us is like, y'all have a lot of AP biology items.

Could we test this? And we're like, yeah, sure. I was a little bit of like, what's in it for us? But like, yeah, sure, that's kind of, I mean, I knew what was in it for me. I just wanted to see what this was. So about two or three weeks later, they said, oh, you know, can we give you a demo?

I was like, yeah, sure. So it was me, my chief learning officer. We get in and they had a AP bio question. And they said, Sal, what's the answer to this? And I was like, okay, it was about osmosis, something like at C. And they said, all right, let's see what it's C. I'm like, oh, that's, maybe it's like, I was like, ask it to explain why it C.

And I know everyone's used to chat GPT and even GPT4 now, but this was before anyone had seen any of this stuff. And I asked it to explain, and it explained it, explained it quite well. And then I said, explain why the other choices aren't right, explained that quite well. I said, and that's when I started to get the goosebumps. And I said, create five more questions like this one. It did it. And that's what I'm like, okay, this is a game changer.

And then they're like, would you like access? I'm like, yep, please give us access. So that weekend, I literally couldn't sleep. And, you know, myself, our chief technology, our chief learning officer, we were literally slacking each other at like four in the morning with, like, things that we had gotten it to do that seemed like science fiction.

The most notable was taking on personas, like acting as a tutor, being able to engage in a Socratic dialogue. And we also were discovering some math errors and stuff like that. We were surfacing that opening it. It turns out that they did have some mistakes in their training data, which we, you know,

so that was our, service to humanity above and beyond the Korkan Academy work, they realize like, oh, yeah, there's some math. But then we started to say, like, look, if we can figure out ways to put right guardrails around this, make sure there's safety, security, we can mitigate the math errors, we can mitigate the hallucinations, this is a game changer. So that's when we started working. Chat GPT comes out a few months later.

I slack Greg Brockman. I'm like, hey, I thought we were under-ending. Like, what are you all launching? And I was like, no, this is just on GPT 3.5, but everyone seems to be taken by this, even the, what we're working on for was so much better. I was bummed at first, but I was like, oh, in hindsight, it was a good thing because it threw out this very imperfect thing that was not designed for education. The education world took it the hardest because kids are going to cheat now, et cetera, et cetera. And then when we launched with GPT4 on March 15th of 2020,

it almost allowed us to directly address all of the concerns that people were already having around chat GPT. And so that's, yeah, that's how we got to where we got to. That's really cool. How did you end up thinking about safety? You mentioned, for example, you wanted to show there were safety around the tool and probably certain answers or certain questions could necessarily be asked. How did you end up approaching that?

So step one, as soon as we saw it, we're like, oh, this could be used to cheat. And so, like, okay, how do you help without cheating? And so that's when we started putting guardrails in the system prompts, so to speak, where it says, don't give the answer, make sure the student does the bulk of the work. It's okay to help them,

but primarily ask leading questions. We have a lot. We've tested a lot of prompts to just get to that sweet spot, and we're constantly tweaking it. So that's the first guardrail. Second guardrail was, you know, this is a new environment, and we don't know where kids are going to take this. So let's make it transparent to other stakeholders what's going on.

So we say, okay, we're going to record every interaction with the AI and make it accessible to parents and teachers if the student is under 18. And, you know, we can debate that for, you know, There's good arguments why older students, there might be some constructive stuff that they could use the AI for that's probably not good to share with parents. But anyway, we took the most conservative stance out the gate, especially for under 18 there.

And then the next one is, you know, we have a second AI that's moderating the conversations. And if it looks like it's going into an unhealthy place, not only will it kind of shut down that threat of the conversation, but it will actively notify parents and teachers. And so that's, those are, and then we are trying this best to do, to do digital literacy as well to students to recognize, like, look, here's what you could use

generative AI for, here's where you should be skeptical of generative AI, etc. And we are feeling increasingly confident that that combination, and also none of the interactions, and this is credit to Open AI, none of the interactions are being used to train the AI. And arguably, there could be benefit in the future for that to happen, but because this is all new frontier and it's all very sensitive. We and Open AI just said, yeah, let's just keep this as kosher as possible. Let's just make this, like, you know, as safe as possible. If you think about, you know, going beyond Khan Academy, you've also spent 15 years generally thinking about education, do you have a prediction for what happens to, like, university

with all of these technologies? Yeah, I think, I don't think generative AI dramatically changes what was already going to happen to universities. I think it's not news to anyone that the return on investment for, for university, especially if you have to take on debt, is, is mixed, depending on who you are and what you do with it. Obviously, we have a student debt crisis, and even if we were able to forgive some, it's not going to solve it for the next generation of students. And so universities are bloated with cost. They're not particularly good at, you know, either informing or preparing students for, you know, in certain cases they are, and other cases they're not, of preparing students for what they need. I think you're going to have a world where

the very elite universities, people join those. Yes, it's almost like you're joining a cast, you're joining a club. So I think they're going to be fine. I think whether or not they can justify the ROI, they're going to be fine. I think, now, there's some interesting things that are happening around, you know, legacy admissions, et cetera, et cetera. So that'll be interesting. I think the community colleges at the other end of the spectrum, they're very flexible and they are very good at thinking about what are the needs? Like, okay, are there industry certifications that could be useful? Okay, we can support students for that. Are there other types of, you know, many badges that could help someone get a job? We can do that. So I think the community colleges that are low cost and nimble and are

able to adapt to the changing environment, I actually think they're going to do fine too. I think that vast middle of universities, especially the ones that are charging, you know, $60,000, when you include room and board per year. And a lot of the kids are graduating with debt that is very hard to even cancel with bankruptcy, and they're not getting the pay to be able to, you know, I tell everyone, you know, all this student debt stuff, they should be holding the universities accountable. Like, where did that money go? Like, who got that money? Right? Someone lent it to the student, and then that went someplace. It went to the university. They took the money. Thank you very much. And now they're just kind of hiding when you have trillion

dollars or two trillion dollars of debt and it's ruining these kids' lives. I think if you, if you, if you, just said, you know what, the universities are accountable for 10% of that. Their tune will change overnight. Overnight, they will start informing students much more about the career outlook if you were to, like, yeah, not everyone's going to become a museum curator. Let me just tell you that right now. And by the way, the kid who you heard about, who just became a museum curator majoring in this major, it turns out that their grandmother, like, built the museum, right? Like, it's a hard job to get. So don't think that this is going to lead there. Just be in And I think the universities, if they were on the hook for even a small fraction of that debt,

would almost not let you major in something unless they felt like you were going to pay back the debt. So I think you're going to, you know, there's a reckoning there. The university costs can't get bloated forever. I think some of what's going on around admissions, the scrutiny, I think it's healthy. And, you know, I'm a big advocate of a competency-based world where it's not how long you sat in a chair or, you know, how long you threw frisbees and went to frat parties. It's what do you know and what do you not know? And so we are working, and I know others are working, to create competency signals, credentials, that if you do it, employers will recognize it.

And it doesn't matter if you did it when you were 16 or when you were 36. You're now qualified for that job. I think that's where employers want to be. And I think it's not going to, quote, disrupt college, but it's going to give college an alternative, which I think is very healthy. I think one of the big shifts that's coming as well is just the type of skills that are going to be relevant in a more and more AI-related world. So if you think ahead, 10 years, 20 years, 30 years, et cetera, are there specific things that, for example, you're encouraging your kids to

learn or that you think that people should really be focused on as they think ahead for their own children, you know, in terms of the skill sets that will be relevant in a generative AI world? A lot and I both have like preschool kindergarten aged kids. So these are active decisions. No, it's a good, good question. It is something I think a lot about. I think there's two, there's at least two metaphors that can help us think about that problem. One metaphor, we can go back to the 19th century, and I'm not just going to talk about,

a lot of people made the metaphor of, like, steam engines and horses and all of that, but I actually think the camera is the best metaphor because being an artist was a real thing. It was really a technical field. You were a portrait painter, and, like, the best artist, you know, you would study for years to be as accurate

to reality as you could, and they could look at how the light moves and all of that, and all of a sudden a camera comes out. and all of a sudden, our artist starts saying, okay, that's the end of art. Like, this thing can capture reality better than, you know, anyone can. But then very quickly, people realize that, no, in some ways, this liberates the artist, that this allows us to think about,

I mean, it's not a coincidence that the impressionist movement kind of coincided with the advent of the camera. People started saying, no, it's not about capturing reality. That's art should be about the impression. Art should be about the feeling it conjures. And it actually all of a sudden led to an explosion of like what art is. It allowed art to come out of that trap of just painting nobility in like these grand scenes and start getting into things that evoke and really challenge us.

I think the same analogy, you know, people are saying, oh, crap, this thing can write pretty well. This thing can code pretty well. This thing can create movies, create images pretty well. So what that tells me is that it kind of liberates the creator to move beyond that. Someone who can elevate and integrate and manage these tools, you know, the other metaphor I would say is imagine if, just as we're saying with Conmigo, every teacher now has a teaching assistant. So now every coder is not going to have an army of coding assistance. Every writer has a writing assistant. So people are going to have to move into the managerial ranks quite quickly and figure out, okay, instead of being a coder, I'm now an architect. Instead of being

a writer, I'm now an editor. And in order to do that, you have to know those skills really, really, really well, arguably better than previous generations. So I think it's even more important that people get not just adequate, but excellent at writing, communicating, coding. I think creativity, because there's so many outlets, someone who's creative who leverages these tools is going to be like unstoppable. They're going to have like godlike power. And that's good. I know there's, you know, there's these debates with the Screen Actors Guild and it's a very sensitive issue. But the way, I think it, just as YouTube, there's a lot of talent that would not have been discovered without YouTube. You would not know about Justin Bieber. You would not know about me. Yes, I just

put myself in the same category, very self-aggrandizing. But, you know, think about how many movies we've all seen that had like a $100 million budget and were horrible, like horrible, like waste of time movies for $100 million. You're now going to have a ton of people be able to create movies with similar special effects and acting and screenplays and everything. And a lot of them are going to be horrible. Even more of them are going to be horrible. But every now and then we're going to discover the Justin Bieber of movie making. And like, that's an amazing movie. And that's some kid, he or she did it on a budget of $1,000, not $100 million. So how do we foster that? How do we give more time and space for that? But I think if you have kids, it's kind of very

traditional advice. They should just get really good at reading, writing, math. I've never been a subscriber of like, oh, there's Google, you can search stuff. You don't have to learn anything anymore. Nope. The more that you have rapid access to knowledge in your head, the more that you are fluent with your mathematics, that you have information and concepts at the tip of your fingers, these tools will accelerate you more than anyone else. And the kid that has to get the calculator for three times seven or has to go to Google to figure out when World War II was, the world is going to pass them by. Amazing. I think that's a really great note in terms of an optimistic view of, you know, it's the same basic skills. It'll be important 20 years from now. And

kids really just need to learn the basics, and you've provided such a great platform to achieve that by. So thank you for all the work you've been doing. Thanks for having me.