Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas - 325 | Alvy Ray Smith on Pixar, Pixels, and the Great Digital Convergence

Episode Date: August 18, 2025

The world is becoming pixelated. As computers and other digital devices become ubiquitous, human knowledge and communication and information is gradually being converted into, and manipulated as, strings of bits. What does that really mean, and what are the ramifications going forward? Alvy Ray Smith is a computer scientist, co-founder of Pixar, and author of A Biography of the Pixel. We go through the journey of how he helped make computer animation a reality, and the implications of what he calls the "Great Digital Convergence." Blog post with transcript: https://www.preposterousuniverse.com/podcast/2025/08/18/325-alvy-ray-smith-on-pixar-pixels-and-the-great-digital-convergence/ Support Mindscape on Patreon. Alvy Ray Smith received a Ph.D. in computer science from Stanford University. He has been a professor of electrical engineering and computer science at New York University, a member of the Computer Graphics Lab at the NY Institute of Technology, director of computer graphics at Lucasfilm, and cofounder of Pixar and Altamira. He is a member of the National Academy of Engineering and a Fellow of the America Association for the Advancement of Science as well as the American Society of Genealogists. He is the winner of two technical Academy Awards. Web site Google Scholar publications Wikipedia

With the new My Lowe's Pro Rewards American Express card, earn three points per dollar spent on Lowe's qualifying purchases for the first six months after card account opening. Plus, save 5% every day on eligible purchases at Lowe's and earn points on eligible purchases everywhere else MX is accepted. Card members get more at Lowe's. Subject to credit approval. 5% can't be combined with any other offer.

Exclusions and restrictions apply. Points subject to loyalty terms at Lose.com slash terms and credit reward terms at synchrony.com slash MLPRA dash terms. Visit Lowe.com slash business credit. Shell v. Power Nitro Plus fuels every drive from the Pacific Coast Highway to the Sierra Peaks with a fuel like no other. It contains active coating ingredients that clean and protect for longer lasting engines. That means more protection with active ingredients for longer lasting engines. Shell v. Power Nitro Plus premium gasoline. Engine performance that lasts.

Chances are you're not far from a Shell station. Find it using the Shell app. Formulation unique to shell. Compared to minimum detergent gasoline with continuous use of Shell v. NITroPlus and gasoline direct injection engines. Actual effects and benefits may vary. See shell.us slash more dash protection for more information. Hello everyone, welcome to the Mindscape Podcast. I'm your host, Sean Carroll. You don't need me to tell you that we live in a digital world these days. We have digital cameras. We talk to each other on digital computers, listen to podcasts, our audio is digital. The whole bit. We've digitized most of human knowledge and our training, our AIs on them. We've had a transition from thinking of the world and representing the world and picturing the world in mostly analog ways. ways that involve smooth configurations of matter that you would, in principle, need an infinite amount of information to reproduce to a digital world where we have finite amounts of data that we can store on a computer, that we can manipulate, compute on, and change the world

around us with all that technology. We see the changes going on. We're really just at the beginning of what these changes are going to bring to us. One example of the digital digitization of the world is the existence of computer animation, right? You can make movies. You can make very popular movies, such as those made by Pixar, as well as DreamWorks, Disney Animation, other studios. And they can be just as emotionally compelling and good storytelling as movies made with human actors. You need both. I'm not saying you're going to replace them. The replacement narrative is a little bit misplaced sometimes. But it's there. It's a form of art, digital media.

Today's guest is Alvi Ray Smith, who is a well-known computer scientist, a pioneer in computer visualization and computer art, and he was a co-founder of Pixar. And what we're going to talk about today is two aspects of this digital revolution, what Alvi calls the great digital convergence. One is, once you have the world in digital form, you can do things to it, right? You can manipulate it. You can think about it. You can create in new ways.

The other way is, are you worried that maybe you're losing something, I literally losing information about the world by representing it in digital form? It turns out that under a very realistic set of assumptions, you are not. And there are mathematical theorems that show you this. And that's a kind of fun thing for Mindscape listeners. But we're going to get there historically. We're going to get there by talking about Alvey's journey from someone who was interested in computers and in art from a very early age in the 1960s in New Mexico,

up to thinking about and developing ways to make computer graphics, ways to animate them, a decades-long quest to literally make the first all-digital computer-generated movie, which of course eventually came out as Toy Story in 1995. Alvey is also the author of a book that came out just a couple years ago called A Biography of the Pixel. And in the book, he talks both about the science of turning the world digital and a little bit about some of the personality, some of the history up until the present day and up until the recent history of making Pixar and things like that. I just love the idea, the sort of conceptualization of what is happening to us in the modern world as a digitalization. and how that changes our capacities, our affordances, our ability to do things, and it can be very hard to predict what will come next.

It's this background that I think is actually very useful sort of for giving us a context in which to think about what comes next in the digital world. And as Alvi says, there are theorems in computer science that say you can't really know what's going to come next. You have to do the computation or in this case you have to live through the history of the future. So that's what we're here to do. Let's go. Alvy Ray Smith, welcome to the Mindscape Podcast. Good to be here, Sean. Let's go. You know, usually I really focus the Mindscape podcast on ideas. And you've had a lot of ideas over your life.

You've written this wonderful book, The Biography of the Pixel, that has some great ideas, and we're going to get there. But I think this is a case where maybe a little biography would be good to set the stage a little bit. Tell us about how you got into the ideas. of computer graphics and ultimately bigger ideas that culminate in digital light. I look back at my life and I can see how I felt I must have been just grabbed by the neck and pushed this way all along. And it just happened.

I grew up in New Mexico and one of the lucky breaks of my life was I had an artist as an uncle. Uncle George taught me how to oil paint. I could sit on his floor. He was the only relative. allow in his studio. Allow. As long as I sat on the floor and didn't utter a sound. And I did that.

And I learned how to prepare the brushes and mix. You know, I fell in love with the smell of turpentine and let's eat oil and all the media aspects of oil painting and learned how to prepare canvas and so forth. So I grew up painting pictures. And I was also smart. So, you know, got promoted to the. top of education system. And long story short, when I discovered computers in New Mexico, I fell in love, as many people have. What year are we talking about? Oh, we're talking the 60s.

Okay. I was at New Mexico State University for my undergraduate work, and this was 61 through 65, somewhere in there. Not a lot of computers hanging around. You could have gone to your life and out pumping into computers. This is when you had to. submit a deck of cards through a window to a holy priesthood. And if you were lucky, it would run. And, you know, I grew up with rockets going off next to Hawaii Sands the missile range with, you know, Werner von Braun came there after World War II and with his V2s to show American rocket scientists how to, how to do missiles. So I grew up thinking it was normal to see rockets going off over the mountains every day. And I heard the first A bomb go off. And so I was surrounded by high tech. And,

Discovered computers and art. You can sort of see how that was going to... It's already there. You're determined. It's going to happen sooner or later, right? It took quite a while to happen, but that's the beginnings. And then I went... Yeah, you went to New York.

Is that right? Well, I went to, you know, I went to graduate school at Stanford. They had a new department there called computer science. You may have heard of this, yeah. And they had a subject that I wanted to learn called Artificial Intellectual. intelligence. This is 60 years ago, Sean. Yeah. There's been a couple of mentors since then. I learned from the original guys, you know. And after a couple of years, I went, you know, Alvey, this is not going to happen in your lifetime. It being understanding how the human brain, mind,

works using the computer as a model. That's what I thought it was. Right. And after a couple of years, I could see they weren't anywhere. In fact, I looked back at it now. It was kind of a silly level of AI. I even taught a course or two at NYU, which was my first job. Okay. And I was working with my chairman, my first chair, my first boss in academia was a fellow named Herb Freeman.

And he was one of the earliest computer graphics people in the world. And I didn't fully appreciate it at the time. I looked back now and I'm so embarrassed. I didn't really appreciate it. He knew I was an artist. He knew I was a theoretical computer scientist, proof theorems, and all that. He thought I would be a magic, you know, the right mix to do, join him in research of making pictures on computers. Of course, I was, but I didn't realize who he was.

He came to me one day and said, well, don't you, you know, Alvey, why don't you come over here and start working with me and my guys? And I said the following of it. Herb, if you ever get color, I'm in. because it was all black and white in those days, and I thought black and white was absolutely boring, right? Yeah, I remember those black and white pictures. So I kind of missed four years of event. I could have left even four years there.

I mean, as if I wasn't on the leading edge anyhow. But then I went to Xerox Park. I broke my legs skiing in New Hampshire, and I was lying in a full body cast for three months, and I got, you know, people rethink their lives. You've heard these stories. I rethought my life. And I said, Alvi, you're making a terrible mistake.

You're advancing right up the academic ladder. No problem. Getting papers published and advancing up the degree, you know, getting fancy and fancy titles, getting grants, but you're not doing anything about your art. That's wrong. What you need to do. And Sean, I look at this.

How could I have been so crystal clear because it was content-free? I resolved that when I came out of the full body cast, I would resign my professorship and go to California where something good would happen. That was the total content of my thought process. Okay. Right. But it did. It did. I did that.

I went to California to Berkeley, as a matter of fact. And my best friend, Dick Schelp, had been encouraging me to come to this place where he had just gotten a job called Xerox Palo Alto Research Center, the famous Xerox Park in its heyday. And finally one day, for reasons we don't need to go into, I said, okay, I'm coming over. I want to spend the night with you. And he says, okay, but you've got to come see what I'm doing in Park. And what he had done was built and programmed a paint program. Color pixels. It was the first color pixels.

Until I wrote my book, I thought they were the first color pixels. It turns out not to be the case, but for all practical purposes, they were the first color pixels. And I looked at it. And it was a paint program. I'm a painter. I was looking for color. He was my best friend. Boom.

He had me hired on at Xerox Park really fast to show off, artistically show off the paint program at Xerox Park. So, okay, so it's starting to come together, right? Then Xerox fired me. That happens, yeah. I went in and I said, well, but why? And they said, well, we've decided not to do color. Okay. And I went, well, wait a minute.

You have, you own color. I mean, I didn't know there was actually some color pixels earlier, but I thought, and it was pretty close to true, they owned all the color pixels in the world. and I said the future is color and you own it all. That doesn't make sense. Alvy, this is what my boss, my Alvi, you may be correct, but it's a corporate decision to go black and white. So I went, okay, bye.

But I had started working with a friend, an artist friend from San Francisco, a video artist, named David D. Francisco, he would come down and we would jam all night. on the on the paint super paint was what this machine was called first color pixel it just but everybody who saw it just blew their minds because nobody had seen color pixels before ever right this was it except some astronauts that i could go into later um we he said alway we need to we should we should submit a grant proposal to the national endowment for the arts to exploit this new artistic medium he didn't have a job i had a job but he needed a job and this is how he was thinking.

I said, well, yeah, he was. And I said, oh, I hate doing grants. I submitted grants to the National Science Foundation. They had to be submitted in 20, 40plicate, 40 copies of this room, you know.

And he says, no, no, no, no, no. That's not the NEE. NIA, it's one page. And you submit your work. I went, oh, okay. So we submitted, we did a videotape, but we submitted that. And then Xerox fired me. I'd lost access to the only machine in the world that could satisfy this grant if we got it.

So we had to, David and I had to go find the next frame. A memory of pixels is called a frame buffer. It was called a frame buffer in those days. Holds one frame. Okay. At video resolution was the idea. We had to find the next frame buffer. So we jumped in my car and tried.

drove over to Salt Lake City because we had heard that this company called Evans and Sutherland in Salt Lake City was building the first frame buffer that you could buy. It was the second frame buffer in the world, and it was a commercial entity. So we get over there. Now, I have to paint the picture a little bit. This is, I've just come out of the late 60s. I've got hair down to my ass. David's got your electric hair out to the, out of two feet.

And we resolved we wouldn't use the word art when we were in Salt Lake City because it was Department of Defense funded. But, you know, we were a dead giveaway no matter of what we said. So they said, folks, guys, we can't take you on here. And all we were asking for was access to a frame buffer. That's all we need. You just need access in case we got this grant. We can't do that. But a rich man just came through here.

from Long Island and bought one of everything in sight. And I said, including the frame buffer? And they kind of looked to each other and finally, you know, I don't know what they got salted, but yes, he bought a frame buffer. So, this probably won't work, but there's a, see that teapot behind me? I see a teapot. The audience can tell. The teapot is very famous in computer graphics.

Right. That teapot in particular is a copy of, Martin Newell's teapot, an Englishman, who was working at University of Utah. And his teapot, he decided to digitize it so that in those early days, there were no databases that we scientists could use to try our rendering algorithms on to see how they look. Did it work? Did the surfaces actually reflect or whatever we were trying to do? And so Martin had carefully hand digitized the 3D points on the surface of his

teapot, and then he would hand his teapot database around. So it became the holy icon of computer graphics, and everybody who did anything always rendered the teapot. And that's, it's now become, I'm a member of the Sygraph Academy, and that's what you get. If you're a member of the Sygraph Academy, you get an actual ceramic version. He says, I'm going, he said, I happen to be going out to visit the rich man on the North Shore Long Island. And I'll call. call you and tell you what I see when I get back. Okay. And he did that.

And basically what he said was, if I were you guys, I'd get on the next plane. This guy had a full animation studio on his campus. All right. This is really hard to describe because his place was called the New York Institute of Technology, NYIT. And at the time,

it was essentially a diploma mill. It's now turned into a reputable outfit. But then it was basically a diploma mill for, kids on Long Island who couldn't get into real colleges. Okay. The rich man's name was Alexander Schur. Nobody's heard of him. He had cobbled together four of the gorgeous estates on the fabulous north shore of Long Island.

This is where, this is great Gatsby land. This is just Pachow Beyond Belief. And we had all sorts of names, Cootville, land we called it. You know, it was the wealthy. And the buildings on those estates, the mansions, were the buildings on his campus. So we did video in one mansion. We did video graphics in another mansion. I lived in another mansion.

David and I, the girlfriend lived in another mansion. You know, just all mansions. I get up in the morning, pinching myself. We talked to away in. That's what I'm trying to say. I did. David took David a year to get in. But I taught my way onto this crazy man, Alexander Scher.

who had an animation studio of the old-fashioned ink and paint type of animation, sell animation. And he basically wanted to be the next Disney. And he thought the computer would be the way of making money. He was really an entrepreneur. So he wanted to get rich. What year are we in by now? This is, this would be the mid-70s. Yeah, okay.

Yeah. Yeah, I think I've got that. right. Yeah, because I exited in 79. And we were in heaven. And he bought us, he bought us everything in sight. So one thing, one day he came to me, this man is really hard

to get across to people because he doesn't, he didn't talk like normal, like you and I are talking right now, exchange, and there's sort of a rhythm to a conversation. He just, he just spewed word salad, we call it. He just Casey Stingle's speech. He just talked. Just whatever.

It was just stream of consciousness. And I remember at first I didn't know what to do, so I just started talking. And after a while, I noticed that if I heard my words come out of, you know, worked into his stream, I must have made, we must have communicated. Success. So I'm not going to go through the full details, but that was the technique. So he asked me, using this odd technique one day, he said, do we have the best computer graphics facility in the world? I said, yes, we do.

Because we did. And he said, how do we stay there? Which is the right question to ask. And I said, well, if you buy me, remember I mentioned the frame buffer? It was the reason I was there was this piece of equipment that held pixels. It was eight bits deep, 256 colors, basically 500 by 500 pixel resolution, which was video at the time. $80,000. $80,000.

$1975 for eight bits. I said, you know, brash kid that I was. I said, well, if you buy me two more of those frame buffers, those 8-bit frame buffers, I can stack them up into a 24-bit pixel memory. And then we have 16 million colors, and then we can do anything. And he wanders off. I'm not sure this has gone through at all, right? Because of just no idea.

Well, several weeks later, he comes back. He said, well, Elvie, I got you five more of those eight-bit thingies. So you'd have two of those 24-bit thingsies. You know, and I didn't know until I wrote my book and kind of sat down and ran the figures that just on my say-so in that weird conversation, he had spent $2 million current money on me. Just boom, just like that. So we had the first 24-bit pixels, which is what everybody uses now, in the world. And we went bananas.

We, art especially, just started creating art just nonstop. And, you know, it was heaven. We were in mansions. We were gorgeous. It was heaven. But it really wasn't going anywhere because we finally figured out this guy. Oh, by the way, the group that had assembled there included Ed Catmull, who would become the co-founder of Pixar with me.

Right. were a few people from University of Utah over there. We decided, you know, what we should do, we should be the first group in the world to make a movie. Complete digital movie. Let's be the first. Now, it took us 20 years to do that. Which we didn't realize.

We didn't realize it. We did it, but it took 20 years and a lot of ups and downs in between. But that's where the idea, that's where the idea started. And, but let's see. So Alex, Uncle Alex, we called him for a strange reason, was, like I said, he had this animation team on his campus. They were creating Tubby the tubba in the old-fashioned way. And he was hoping, in his words, he was saying, can you make tubby dance using the computers? We couldn't.

We could already do anything. because computers were so slow still. Eventually, the movie came out, and I remember sitting in a screening room in Manhattan. We were all horrified by this movie because anything they could be wrong about a cell animated movie was wrong. There were shadows under the ink lines. There were dust on the frames.

The story sucked. It was just everything was bad. We're wincing. One of the animators, young animators, with whom we'd be friends and came up and said, I've just wasted two years of my life and he was gone. You know, so we knew that this was not the guy

that was going to get us to the first complete movie, right? In California, staying compliant means watching the state laws and the city rules at the same time. And no wonder it feels overwhelming. Meal breaks, rest breaks, wage rules, constant updates, it's a lot. And that's why Southern California

businesses rely on Guardian HR. They're local in L.A. And they understand this community, and they help you stay compliant to avoid costly missteps. You get accurate payroll software and a real HR expert who keeps you ahead of issues. Get your Southern California business protected at GuardianHR.com. But sorry, you were clearly interested in some intersection of art and computers very early, but was it his uncle, what was the name? Uncle Alex? Alex. Alexander Scher. Was it his initiative that turned the specific form of art toward video, toward movies?

I guess you could put it that way. I mean, it never was that clear. He was making a movie, and he wanted us to help him make that movie. And the result of that was we were surrounded by an honest-to-god, Hollywood-educated film production crew who knew how to do it. And they showed us all the nitty-gritty of making an honest-to-goddye. honest, the got sell animated movie so well that we were able to digitize that process for Disney years later.

Right. It's one of the things we did to make the relationship with Disney work. So in a way, you could say that, although it was, we kind of knew that, I don't know, I looked back and we certainly didn't conceive of a Pixar type movie yet. We would have been happy with a 2D movie. Yeah, a Disney type. as long as they're completely done on a computer. So drawing, he was still using hand-drawn, you know, animators,

still drawing pencil on paper, and then we would digitize that. Nope, that's disallowed. You've got to do everything on the computer. Yeah. Okay. That was, that's the rule for this particular project. Anyhow, about that time, we're realizing this guy hasn't got it, even though this is heaven on earth.

We were just treated like gods, and it was wonderful. George Lucas knocked on the door and Francis Ford Coppola. Two separate days. Coppola, not themselves, they're people. Sure, sure. One day, Coppola knocked on my door. His people knocked on my, I knew the guy who was representing Coppola. I distrusted him.

And when I realized, if this is who Copla has, this won't work. Right. Because we're going to cost a lot of money. Somebody is going to have to be strong and steady on the finance part for a lot of years because this is a biggie. And even though I think Oppel is by far one of the greatest directors of all times, he was already famously impractical about money. Yeah. He would come. He would, okay.

Francis, I remained friends over the years, but that's a separate story. George, on the other hand, we had seen his movie. We had rolled in, again, I jumped in my car, rolled in Manhattan and seen this movie Star Wars, which blew us away like it did everybody. And so we always thought it was going to be Disney that finally came and did us, paid for us. And Ed and I would make secret pilgrimage to Disney every year. We just were so sure that it was Disney. we would just make sure they understood where we were and what we were doing and what we could do for them. It never worked because Disney was run by a football player during this period.

His name was Ron Miller and he was an L.A. Ram and his qualification was he had married Walt Disney's daughter. There you go. And basically they would reissue Cinderella or Pinocchio every year or so. But the company was just. But on these very big. visits that Ed and I would make, we got to know the underlying technical team. They knew what we had. They knew that Disney needed us, but they also knew that we were going to just hit a dead end as we went up to the VPs. And we did. They just wouldn't listen to it. But that's when I met Frank Thomas and Ollie Johnson and some of the great old animators. And, you know, we just learned the ropes. And so, much to our surprise, it wasn't Disney who knocked. It was George Lucas's real estate man, it turns out, knocked on our door.

It said, George is looking to digitize Hollywood. Well, that's a noble ambition, and we thought that was the right thing to do, and we could do that. The big mistake we made at this time was to think that he wanted us in his movies. He wanted us to make pictures in his movies. we just assumed it because that's we were about making movies right i already had a piece in museum of modern art in new york it was that this content was what we were about yeah we can program and build hardware yeah we're absolutely willing to do that to earn our keep and so got hired on at lucas film and i started i started hiring the best computer graphics people in the

world. Everybody assumed we would be in the movies. Everybody wanted to be in the movies. It's just kind of a natural instinct, right? So I'd get these people who are just the crim to the ground in the technical aspects. And also animators started showing up, wanting to be part of it. And I usually, so like Brad Bird was one of the first to show up. Sure. And he was the funniest guy I'd ever met. Just had me in the stitches every moment. He just was, it was just nonstop. He was one of those naturally funny people. But he was also a sensitive artist.

And in those days, I said, you know, we were at Lucasfilm, but we were in a converted laundromat. And it was freezing cold because the machines had to be heavily air conditioned. They broke all the time. They were slow as I'll get out. We would put up with for it because of, we understood Morris Law, you know, that we're going to be saved by Morris Law, basically. Yeah.

But how do you, I just couldn't do that to, to Brad. I says, you know, we're just not, we can't do that yet. You don't want to be here, not yet. So, but, you know, the animators are starting to get the idea that this new thing was coming along. And they wanted to be in there. And so I got this team together and waited for George to come around to ask us to be in his next movie, George Lucas. He never came. And all of a sudden, I went, oh, my God, he doesn't get it.

He doesn't get what he doesn't know what he has. He's got the hottest team in the world and he doesn't know it. What did he want from you? Hardware and software. Hardware. To digitize, he wanted us to digitize audio production, video production, and just, you know, keeping track of the books. And he also wanted us to, do you know what an optical printer? is, I think that's the machine that loses most people. Optical printer mixes film from two

different cameras together into one piece of film. It was an art form, basically, at the time, and he wanted us to digitize that process, which we did, by the way. We did all of this. We came up with an audio editing system, a video editing system. We did logistics. We did a digital optical printer. We did all that. But what we really wanted to do was make content. And you knew at the time about Moore's Law and you knew you could sort of sketch out what the requirements would be to do what you wanted to do. So you could sort of tell eventually you'd be be able to get there even though we can't do it now. Yeah, it's just everybody knew that computers are getting faster all the time. It was just part of the structure of the universe. People

already knew it. I can't remember whether we called it Moore's Law yet or not. I imagine we did, frankly, but we just knew that every year was better than it was last year. So you needed somebody wealthy behind you who would keep buying the latest of everything. So here was George Lucas and he was apparently doing that. He set us up really well and let me hire all these amazing people. So what saved our bacon this time was Star Trek 2. So Paramount came to industrial light and magic, which is the special effects branch of Lucasfilm, and asked for special effects for Star Trek 2, The Wrath of Khan, their movie, the Star Trek franchise. They asked ILM if they could do some of this newfangled computer graphics stuff.

and my Olympus said, well, no, we don't do that. I think the new guys next door, that would be us. I think that's what they do. So they call me over, and, you know, they sketched out this scene requirement they had. And I listened to it for a while, and I kind of sat back front. I said, do you guys have any idea what we can do this day and age with a computer? No, no, they had no clue. And I could tell from what they proposed, they had no idea.

I said, okay, I'm going to go home tonight and draw a storyboard and come back tomorrow and present it for something we can actually do now, utilizing all the best talents that I have. You know, I had a guy on fractal, a guy on bump mapping, a guy on texture, man. You know, I had all the, I had these great rock stars. So that's what I did. Up all night, walked back in with this hand-drawn storyboard and got the job for a 60-second shot. We call it the Genesis demo. I remember the scene. Oh, okay, okay.

At the time, it was a mind-blower. It looks kind of crude now, but at the time it was a mind-blower, we were extremely, extremely proud of it. So I told, I pulled the team together and I said, look, this is it. This is our big break. This is the first time we'll be on the big screen and part of a movie that's going to make a lot of money. There had been computer graphics and movies before, but they failed. Looker is it? You ever heard of Looker?

Probably not. Nope. You have? So what we're going to, and this is a 60 second. piece. So we're going to make Paramount really happy. We'll execute the shot as promised. It'll make the Trekkies in the audience really happy because it'll be exciting. You know, fifth of narrative, they'll make perfect narrative sense. Everything here will make narrative sense. But what this really is, is a 60-second commercial to George Lucas, so he'll know what the hell he's got. And I knew one thing

about George. I look back at it and I still don't quite know why I knew this, but I did. Probably because I've been watching movies with him and I just picked it up. He always, always, is aware of the camera. Now, if you go to the movie and try to stay aware of the camera, then the director has failed because you have not been sucked into the emotion of the piece. And apparently George could do both. Because, you know, but I knew he knew where that camera was. So I said, we're going to put a camera shot in here. Again, it won't be gratuitous.

It'll make perfect narrative sense. But it'll be something that no real camera could possibly do. And it'll blow his socks off. And that's what we did. We put a very elaborate. So there's a, the camera's attached to a speeding satellite that has to turn around and look at this planet. And it has to track the flame coming over the limb of the planet. And then it has swoop around to all sorts of stuff.

And finally, now it's doing a complete 3D moves that no real camera could possibly do. That's what we knew George would see or I assumed he would see. All right. The day after the premiere, George, who had never spoken to me, hardly at all. I think he was shy. He stepped one foot in my office and he said, great camera move. He'd read him correctly.

He got it. And he had us in his next movie, just a tiny piece, which was Return of the Jedi. And more importantly, he told his best friend, Steven Spielberg, about us.

And Stephen had us in his next movie, Young Sherlock Holmes. And then the word started to spread. Right. But we, you know, we didn't want to be effects animators. We wanted to be character animators. We wanted the characters to come alive, just as Walt Disney had taught us on his TV show when we were little kids. He showed us how to do it.

Well, he showed us how they did it. But, you know, let me just say, like, that's a non-trivial statement about you and the people you were working with. Plenty of people would have been just happy being special effects animators, but you wanted to... And there were a lot of companies were, yeah, yeah. And you wanted to make characters and tell stories. Yeah, yeah. Because we had been, you know, I grew up watching, you know, Cinderella and Pinocchio and just been in love with these wonderful movies. Took you out into a completely different world.

And then Disney himself, old Walt, showed us how to do it, you know, draw and ink and paint, so forth. But then George and his wife, Marcia, got divorced. All right. And in California, community property state. Overnight, George loses half his fortune. And we're an expensive group. So I went into Ed, my partner all these years, and I said, Ed, we're going to get fired. George never really understood who we really are.

we cost a lot of money. He's just lost half of everything. He's going to fire us. And it would be, and I use this word sin because Ed was a good Mormon, and I grew up Southern Baptist, so we know these words, right? I said it'd be a sin for this world-class group to disperse. And I should add at this point that we knew we had a coal fact under our belts at this point that it was going to take five more years to get Morris Law to where it needed to be. to make a movie. Okay.

So we had to hold this group together for five years somehow. And it can't be computer graphics because that's not here yet. Let's start a company, I said. He agreed. Now, this is two nerds talking to each other, right? We're not. Not captains of industry. No, not at all.

We had always been working for the man. Yeah. And so we went across. the street in Marin County and to a bookstore called a clean well-lighted place for books. I remember it well. In Larkspur Landing, Moran County. I bought two books. Ed bought two books, how to start company books, how to reprofit and loss sheets,

how to do business plan, you know, really basic stuff. We had no idea what we were doing. But the damnedest thing is it worked. We wrote a business plan. We decided, okay, the only thing we can do to support a group of our size, which is about 40 people now, is hardware. Doing software, doing commercials didn't pay. Doing software didn't pay.

The only thing that could possibly pay for this is hardware. Well, we had built, I called it a digital equivalent of an optical printer earlier for George. We had built this special purpose. computer and we so i said we could just take that prototype like they do in silicon valley and turn it into a product and sell that we already have the prototype then we've got the team that did it so it's just sitting there so that's what we decided to do run up the business plan and uh then our troubles began because no you've got to find Capital when you start a comedy, right?

The hardest part. Well, Ed and I talked to every venture capital firm in Silicon Valley. San Hill Road's famous for being the foes. I have, okay, so we were lucky in that we were sexy. We were part of Lucasville. So we got it in the door, whereas a lot of entrepreneurs can't even get past the door, right? Well, we got past the door, but 35 of these outfits turned us down. We talked to everybody. They turned us down. We were in the wrongs. They didn't, building, you know, we're

presenting ourselves at a hardware company with this, but we had no experience. We already had the prototype bill. We were past seed money. You know, it just, we didn't fit. 40 people. It just didn't fit. And they said so. You know, We camp, it's interesting, good luck. So the next idea was we would form a strategic partnership with a large corporation. So we started talking to large corporations. And we talked to eight, who all said, not a chance. And then two said, okay, General Motors, the largest corporation in the world at the time,

and Phillips of the Netherlands. decided to go together. If we would do renderings of automobiles for GM and medical imaging for Phillips, they would finance our company. That almost happened. That got right to the final stages of negotiation in a Manhattan skyscraper right of Grand Central, I remember it's one of the most incredible days of my life. Four parties, 20 attorneys, just, you know, just it log jams, stress headaches.

It's just an awful day. But at the end of the day, everybody's smiling and shaking hands. And in the business world, that's a done deal. Right. But not this time. I mean, usually overnight, the attorneys write up what just got said. Right up the details. And there's a signing.

Okay, it's done. No, not this time. Because at the same time we were doing this meeting, H. Ross Perot. Remember this guy? I will. H. Ross Perot? It was his branch of General Motors we were dealing with. And he was uptown insulting the board of directors of General Motors at the GM building.

And overnight, it broke that, you know, that basically they had split. GM was getting rid of H. Ross Perra. Well, we were the baby in the bathwater. That was us. Right. It was dead overnight. So we're now 45 deals. You know, Ed and I have tried 45 different ways of getting financed.

That's a lot. And everybody's saying, no, we thought we had it and didn't. And, um, so Ed and I, in the limo, going from Manhattan back out to JFK to fly home, and we're just frantic. You know, now what do we do? When we came up with what I call our Hail Mary pass, neither one of us being sports guys, by the way, that was just, yeah. Let's call Steve Jobs.

Now, Steve, we had already met Steve. He had been fired from Apple, and he had called him. told Ed to me down to his mansion outside of Woodside, California, which is near Palo Alto. And I remember sitting on the grass and he proposed to us that he buy us from Lucasfilm and run us as his next company. And we went, no, we want to run our own company, but we'll take your money. And he agreed. Okay.

He agreed to that. So, but, but every deal had to be cleared through. Lucasfilm because they were getting a piece of the action. Well, Lucasfilm thought they had General Motors Phillips on the hook at this wonderful valuation, which turned out to be about twice the valuation of what Steve was offering. So they kind of more or less laughed him out of the office, so to speak. All right, but at this point, GM Phillips has failed. Ed and I say, let's call up Steve and say, Steve, everybody's at the end of their rope.

Just call Lucasfilm and make exactly the same offer again. Same valuation being half of what, okay. Otherwise, nobody's getting anything out of this, right? Lucasfilm loses everything, and we lose our company. And he did. And that's how we got financed by a hardware guy. We thought who could help us as a young. hardware company, start this company.

Were you at the time still thinking in the back of your minds that you're presenting yourselves as a hardware company, but you're going to keep alive the dream of making a movie? Oh yeah, we made it clear. There was always be this group. We had John Lasterarty, the animator. We had the guy, you know, we had our core team there. Everybody was aware of it, that that's what we really wanted to do. But in the meantime, to make money to pay for the corporation hardware, marketing, distribution, and all that. Well, and we had this hard hardware guy we thought on the board.

So Ed and I and Steve Jobs are the board. And we failed. As a hardware company, even with Steve Jobs, we failed. I think three times, it depends on how you countermeasure. But when you run out of money and you can't pay the bills, in Silicon Valley, you're dead. Except Steve did not want to be embarrassed. by failure after having been booted from Apple.

So he would tear it and me a new one, but he would write another check. He would take away equity. And from us, from the employees, originally the employees owned 30% of the company. So he would take that away, pieces of that away, and write a check. And he did that three times, I think, until he had taken away all equity. He owned it all. He finally did buy. A lot of people say he bought it.

No, he didn't buy. He did buy it eventually from the employees over a course of several years. And we didn't have anything. But we had our company. We kept it together. And we kept it together for five years. Warslaw did its thing. At the almost right on cue, Walt Disney shows up.

Now, we had already made a great relationship with Disney with a project called Caps. But they said, let's make that movie you guys always wanted to make. We'll pay for it. You do it. So what year are we now? This is this has got to be like 84, maybe 85 somewhere in there. Oh, okay.

No, wait a minute. I'm wrong. That's pretty early. Pixar started in 86. So this must be 91. 991. 991. Yeah, yeah, yeah.

Okay. Because it took several years to actually execute. the movie once they started funding it. Okay. And the movie came out in 95, I think. So Disney saved Jobs' financial ass because they came in and paid the bills. They also knew how to make movies. Yeah, they actually wanted you to make a movie. Yeah, they actually wanted us to do what we always wanted to do.

And Steve, to show off his business smarts, which he does have a lot of, decided. Okay, so Toy Story gets made. It's in the can. Pixar takes it to Manhattan's, and the critics see it, and they go nuts.

Nobody's seen anything like Toy Story before, right? Steve's brilliant move was at that point to take Pixar public based on nothing but having that toy story in the can. We had no money. But it was a brilliant bet. And he became a billionaire overnight from that bet.

Okay. And when you say, you know, you were thinking since the 70s about making a fully digital movie, the movie that was eventually made was Toy Story and it was obviously an enormous success. But you weren't thinking of Toy Story in the 1970s. You just wanted to do it. Not at all. No, we had no idea that we were going to be Pixar or that that would be our look, you know. Or the plot or the characters, you know, all that.

None of that. Most of that came out of John Lasseter's head. He was this amazing creative animator that I hired along the way. Again, it was one of those lucky things that happened. I told you about how we visited Ed and I would go down to Disney like once a year. even while we're at Lucasville, we're still going down to Disney. I'll keep that relationship alive. Somehow, they've got to figure in somewhere.

And I'm leaving out a whole Caps project, but on one of these visits, a young animator named John Lasseter is there. He's working for Disney. He's really excited to meet us. And he says, hey, guys, you want to go down into the archives? And I says, yeah. Yes, yes. So he takes us down into the archives.

He says, well, what do you want to see? And I said, anything? And he says, Alvey, anything. Yes. I said, well, I want to see the Preston Blair's dancing hippos from Fantasia. Okay. He says, okay. And he goes and looks up on a chart and pretty soon he's got this folder open and he's

thumbing through Preston Blair's original drawings of figure name was Petunia, the Hippo, something like that, dancing. and I'm just flipping out, right? And we're bonding, right? So there's some serious bonding going on at this point. And then he said, okay, John says, now what? I said, well, anything. He said, Alie, yes, anything. I said, okay, the drunk elephant scene from Dumbo.

Boom, we get that. You know, we're off and running. But we couldn't touch him because he was working for Disney. So a few months later, Ed's down on the Queen Mary, which is the, original Queen Mary, which is docked in Long Beach permanently docked, and it's a convention center now. And he's at some meeting down there. And we're doing our daily business talk. And he said,

oh, by the way, John Lassiter just came by and he's no longer at Disney. I said, it, get off the phone now and go hire him, which he did. Of course. It was. And the rest is history, yes. So he came up, he came up and saved. I had, we had decided to do a piece at, the 1984 sigraph to show people we did character animation and not effects. And I was assumed I was going to be the animator, foolishly, thought that. And I had done the storyboard and was executing away on it. And then John, you know, we got John. John looks at my storyboard. He says, mind if I make a few suggestions? I said, no, no, that's why you're here. Well, he softened the character and, you know, he just made it, he made it, he made it

where he saved myself. He saved me there on that one. And we took it, this is all happening at Lucasville. We took it to Sigraph. It blew Sigraph away, just 5,000 screaming fans who knew they were seeing the future. Because this was the first glimpse at the so-called Pixar look, right? Right. And George Lucas was in town, and he came.

This was Minneapolis, and he came because Linda Runchy, Statt was in town. And they were a thing. So Ed and I and Linda Ronstadt and George all went out to dinner. Then we got after there, we got into a limo and we went and down into the bowels of the sports stadium in Minneapolis where Sigraph was being held. Sigraph's the annual computer graphics conference. I think about 40,000 people are attending by now.

Huge thing. And the film show had like 5,000 people in the audience. night. And we knew when we showed this piece to this audience, it was going to blow their, you know, it was going to blow their minds. So, and George is going to see it. Finally, he would see, he, it was expressing interest in what we were doing. And he would see it recognized by our peers in an astounding way. But, you know, he didn't. He, he just, he saw exactly what he saw. He couldn't induct to what this meant for the future. He saw, there was an unfinished, one scene was unfinished that still had line drawings in it.

And, you know, I look at the piece now, and it, yeah, he's kind of crude a little bit, you know. But. And luckily, he didn't say so. He hated it. We found out later. He just hated it, but he didn't say so. He would have broken our hearts if he had said so.

And, you know, again, Why people didn't One of the fun things You look back at this history is They let the people with them Steve Jobs by this time It probably owns nearly everything And he

At Pixar He never got rid of the team That was John Lager's team That was making short subjects He kept it on Even though he had no concept It was not his goal to make a movie He didn't know anything about movies

But He could have just actually but he didn't. I guess he knew that if he asked that we were out of there because that was our whole reason, right? Yeah. I mean, maybe this is a good time to step back a little bit. And in the book that you wrote, you sort of ended the book with what we've just been talking about, these stories of how you actually implemented these ideas.

But you start the book with a very big picture perspective on the digital convergence that we're coming in. I would love to talk about this because this is what my book is really about. Well, that's what I really want to hear about. I mean, you certainly, you convinced us that you've been through the wars here, and you're the one to think about this in a careful way. And so, I mean, how are you thinking about, well, it's your choice. We could talk about the big picture of the digital convergence,

or we can spend a little fun time talking about Fourier Transforms and the sampling theorem. I would rather talk about the digital convergence. Good, let's do that. And if there's any more time, we can talk about the sampling theorem. Great. Which is a beautiful thing that I love to present.

But what the book is about, and it's Cecil right from the beginning, I think, is that something happened about the year 2000. It was huge. And nobody paid any attention. I call it the great digital convergence. So I wrote the book to celebrate this major change in our ordinary life and celebrated because it was big.

Now, the way I presented in the book was it's the moment when all old media types disappeared and got replaced by the bit in the form of the pixel. No more film, no more videotape, you know, no more analog anything. No more analog audio recording. it all became digital. All AV, all audio visuals became video. But what Allison, my wife Allison has helped me put this in the larger perspective. She says, you know, Alvi, it's even bigger than you said in the book.

Everything became bits. Everything. The whole world of knowledge became bits. Sorry, as opposed to analog. I mean, you're drawing a distinction. Yeah, like, well, yeah. Because, for example, libraries became bits instead of printed pages. All.

Not, I mean, there's still things that haven't been digitized, but essentially we now have access to everything or could if somebody just gets around to digitizing. All right? And Moore's Law is still ripping away. This amazing supernova explosion of computational power. the digital conversions meets the supernova Morse law and gives us what? Well, we're in this AI revolution that's just blowing everybody away right now.

That's one of the results. Now that computers are fast enough to go looking at every piece of fact that's ever been made, they can find correlations that we never suspected we're sitting there all along. Chat GPT wise, okay, okay.

It's a story of turning artifacts into information in some sense. Like I think in the book you mentioned, you know, a painting used to be a painting. Like you had paint and you had canvas. Yeah, you had the medium used to be important. And all of a sudden we said,

no, you don't need the medium. It's the picture is independent of the medium. The information is independent of the medium. What a shocker. But that's the way we're writing right now. And I'm continually blown away by it every single day as people come to me and tell me what they can do with an AI program. Like I have a, I have a friend in the financial business who says, oh, I don't write code anymore, Alvi.

I just, the AI does it. And I says, you mean good code? He said, oh, yeah, difficult code, elaborate code, and it's correct. I just I said so you mean all the all the software guys at your company are going to go away and he says yeah

they're not needed anymore this does a better job okay now I'll get stories like that and I've heard about that it's even in the end of my book because while Alice and I were in Cambridge

one of my colleagues John Bronskill who made his initial money from Photoshop filters. In other words, he was a pixel packer like I am. He was going for his defil at Cambridge when we were visiting,

and he came rushing across when he saw me and said, hey, Alvey, we don't need to program anymore. Now, this is 2015 or so. Okay. All right, kind of 10 years or so ago? Yeah, more. Somewhere in there. I think it's more than 2015.

But, okay. Anyhow, somewhere in there, he says, we don't need to program anymore. And I said, what do you mean? He says, read this. And he slipped me the, what I call the zebra's horses paper, which comes from Berkeley right here, the AI guys here at Berkeley, who showed that if you trained a neural net in a certain way, you could give it a picture of zebras, and it would give you back that same picture where every zebra have been replaced with a horse or vice versa.

And I went, wait a minute, that doesn't make, that's an ill-posed problem, isn't it? I mean, what's a horse to a computer? What, you know, he says, Alva, that's what I'm trying to tell you. We don't know how it works. We don't know how it works. It just does. And there's a point where I went, oh, my gosh, I always thought I would understand as we got to closer and closer to, you know, so-called intelligence, that I would always be able to

understand each advance. But of course, Turing told us in 1936 is, no, no, you won't. We can't even tell by looking at a program whether it's going to stop or not. That's been known for decades. And to think you know where a program is going to go, no. I'm telling you right now, mathematically, you can't know. That's where we are, I think. I mean, it's a deterministic process, but I think, again, as you say in the book, the own only way to know what the outcome of the computation is going to be is do the computation. That's right. In general, it's each step is deterministic, but the outcome is it, is not determined.

It's interesting. I mean, it can, it's not predetermined. That's what I should say. Yeah, I mean, you're married to a philosopher, so you know this, you know, careful distinctions. It is deterministic, but there's, you can't know it. So it's kind of who cares if it's deterministic or not, you got to actually do it. Right.

Exactly. Yeah. Yeah. So what's that mean? that means you just jump on the wave and write it because see what happens. Well, I've been doing that all my life, right? I've been writing the Moore's Law. I was there in 1965 when I came to Stanford, Moore's Law Factor was one.

That was the year that Gordon Moore explained his observations, thinking it would last for 10 years. Well, it's lasted all my life, and I don't see any end in sight. the one dimension I didn't pursue in my book, and I would have in a rewrite, is, oh, by the way, once these computers get very tiny because of Morris Law, you can also explode how many computers there are working in parallel. So that's sort of the Nvidia explosion. It's yet another dimension that makes the exponentiality even bigger than Morris Law. But once you get into exponentials, it's all, it's all mind-blowingly beyond. human beings so you know you have this in the book your version of Moore's law your way of stating it is kind of a charmingly informal one where you basically say everything good that a

computer can do or increase in play in order magnitude every five years yeah and order magnitude is being what a human's capable of that's about as far as we're capable of seeing into the future is one order of magnitude I can sort of I mean I can let's let's take three orders of magnitude, a factor of a thousand. Well, I can tell you how much faster everything will be when it's a thousand times faster, but I can't tell you how different the world's going to be. I couldn't tell you a thousand, a factor of a thousand ago that it would feel like AI does now. Nobody saw this coming.

So I guess that's going to complicate my next question, which is if you think that we are through this great digital convergence 25 years ago or so, I mean, that's still young. That's not that long. No, it just happened. It just happened. So what are the implications of this? How do you think, how do you think we should be conceptualizing it? That's why I bring in Turing.

He says, you can't know. You can't know. You can try your damnedest to try to push these, push these, I don't even know what to call them. I don't like saying AI because it sounds like there's a machine that's intelligent. And it's just not. It's a bad terminology. It's a bad terminology. It's a communication system that, you know.

Like Allison points out, the library at the University of California holds tremendous amount of information, tremendous. But you never think of the library of California as being intelligent. It's just, okay. All right. Or the card catalog, which lets you. access it, you don't think of it as intelligent either. Anyhow, here are the things that I'm

having trouble foreseeing. I don't dare try to forecast what's happening next because I don't think we can know. Nor what it's going to feel like, because it sure feels different now than it did back then. Yeah.

And even though we knew Morris Law would get us here, we didn't know it would feel like this. I keep looking at what it is that made Pixar special. And that's the creativity of the technical geniuses and of the artistic geniuses. And I can't see, I want to believe that machines can get there, but that's religion. I just don't think there's nobody's show me any possible way of getting to the creative part.

It may just be my desire to hold on to something special for humanity. But you mean you were there? I'll hold on to as long as I can. I just don't. And, Sean, I, you know, I think I'm a machine. I don't see any reason I'm not a machine. So if I'm a machine, there must be an explanation, right? But the levels we're at now aren't the explanation.

Good. Okay. I mean, the way I think about it is the large language models, we've had such enormous success with or deep learning machine learning more generally is trained on things that are the products of human creativity

and they're very good at remixing and remembering and distilling and searching and all those things there's no reason they can't be good there's no reason why computers can't be good at being creative but that's not the things that we built right now well also definitions It's a matter of definition.

In my book, I often solve the historical problems by just defining terms. Like, I couldn't believe that I didn't know who built the first computers when I sat down to write the book. As a word computer, people, if it had vacuum tubes in it, it was an electronic computer. I went, no, it's not. That's not a computer. Well, once I define the terms, then you can just pick off who did what first. It was easy. But you had to define the terms.

Same with movies. Same with intelligence. Somebody give me a definition. Just saying intelligence doesn't say anything. In your own experience, I mean, you've been around a lot of sort of technically creative people, but also artistically creative people. Are these different things that we just happen to give in the same name to, or is there a relationship there?

Well, there are different people, that's for sure. I think it's the same thing, but it's applied differently. Probably takes a lifetime of preparation to make the, artistic leaps and a lifetime of preparation to make the technical leaps. But, you know, one of my proudest achievements at Pixar is a culture that honored both of them. Right. Because it would not work if one side looked down their noses at the other side. And by the way, most places I've been, that's the case.

Like Microsoft, I work at Microsoft. Heaven on earth for programmers. But they would look down their noses at people who couldn't program. oh, if you can't program, then maybe you can do this marketing or art or that kind of stuff. And graphic arts houses, just the opposite. You're a creative or a technoid. Right. No, no, no, no, no, no, no.

We never let that happen. So, you know, everybody, not only was it equal, you know, equal salaries, vacations, titles, all of that, but dignity. you had to appreciate what the other person was doing because the whole company depended on it, that kind of appreciation. And I think that's what Pixar is and was. Well, the idea of the digital convergence, the idea that things have gone from being analog to being digital

and that opens up new ways of dealing with them and manipulating them and so forth. Some people are going to say, okay, but the digital representation of the information is never perfect. It's only just an approximation. or something like that. No, I don't believe that.

I know there are true believers who think vinyl cannot be, you know, nothing can be as good as vinyl when it comes to audio. I just think that's wrong. The theory says if you do digital representation correctly, and nobody does, by the way, but if they did, they could simulate vinyl. But it's probably cheaper just to buy vinyl if you want that. Probably.

And the same goes with With with with Well with any recording medium You can you can just say The original you know film is just better Because you had this It's got this graininess to it Or that was okay if that

Becomes your definition of good We can add that noise back in if you really want But I don't think anybody's going to bother right Well I think this is it's late in the podcast So we can you know indulge ourselves a little bit and I really want the audience to understand your conception of what a pixel is. Because this is, it's interesting because you clearly think about pixels differently than I do. I used to think about them the same way everyone else does as little squares on a grid.

And you're teaching us. Pixels are not little squares. They never have been ever. They've never been little squares. Everybody suffers from that. Yeah. I mean, you can't see a piece. pixel just to start with. That kind of sets people back. What am I looking at right now and I'm

looking at Sean on my screen? Well, you're looking at display elements. Well, display elements are these little gloy pieces of light that aren't, they have a shape. They're continuous analog blobs of light. They're controlled digitally on a grid. They're driven by a pixel, but they're just little blobs of light and those little blobs of light intersect one another to form of, the picture I'm looking at is, is not broken up into little anythings. It's a continuous surface. Swimming by a picture. Now, people say, yeah, but if you zoom in, you'll see the pixels.

Well, that's a dirty trick. It never has been true. If you zoom in, every app has a zoom feature. And I think this is one of the reasons people think that they're looking at little squares is because if you zoom in using the zoom feature on most apps, you eventually get to the point where you're looking at an array of little squares. Oh, it must be little squares down there because I zoomed and there they are. But no, that's a dirty trick. It's a real simple hardware trick to replace each. Suppose you want to zoom by a factor of 100. Just replace each pixel

with a 100 by 100 square array, and that gives you a picture that's 100 times bigger in both dimensions. But what you've done is to replace a little pixel with a 100 by 100 square array of pixels, which guess what? It looks like a little square. What a surprise. But it's not a picture of a pixel up close.

It's a picture of 10,000, what is that? 10,000 pixels. So what is a pixel? It's a sample. So a sample is, so I like to think of a picture, like when I'm looking out of view right now, is a crumple sheet. So I'm going to, you usually look at it from the side, but if you turn the sheet sideways and look down on it, the heights of the crumples in the crumple sheet are the brightnesses of the picture at the various points. at an infinity of points, because there are an infinity of points in this crumpled sheet. Now, you take a bed of nails, where the nails are equally spaced in both dimensions,

and you let this crumpled sheet settle onto this bed of nails, and then you throw away everything on the sheet except the points that land on the nails. Those are the pixels. the samples of this continuous surface on a regularly space grid. It's just a number taken from the continuum. So it's an abstract informational concept. It's not a physical. It is.

It's just you can't see it. It's just a number stored in a piece of memory, right? All right. You can't see it. So, you know, like I like to say, I hold up my phone. I say, I've got billions of pixels in this phone. but I can't see any of them. You can't either.

On your phone, you probably have billions of pixels, too. You can't see them. They're there. They're in a file. Because that's how pixels are stored. They're stored bits in a file. To see a picture, you click on the file name or whatever, and suddenly there it is on your laptop or your cell phone.

What's just happened? Well, like very fast computation. has just happened, where that array of pixels has been reconstructed using the sampling theorem into the image that it originally represented. And the sampling theorem is a mind blower, but it basically says if you do that crumpled sheet on a bed of nails trick the right way, you can throw away all the information. It seems like you're throwing away an infinity of information between the nails. and not losing it.

And not losing it is the mind-blowing mathematical trickery. You don't lose it. Those little blobs of light happen to be the almost exact theoretical shape needed to reconstruct that the theorem requires. And at that point, you have to go off and do the details of the theorem. I don't want to go into all the details of the theorem, but maybe a detail or two would do us good here. Because it fascinated me in the book discussion because you, actually had to care about the difference between countable infinity and uncountable infinity. Oh, you want me to do that? All right.

I do. Okay. So the sampling theorem, by the way, there was a fight over who invented the sampling theorem. And I came to a solution in my book. It was a very surprising solution. I was convinced by your story. It was a Russian. Yeah, it's a Russian commie yet.

Okay. He was a really smart guy. he said you can sample, as I explained, the sheet on the sharp nails. And then later, you can, if you put a certain shape on each of those nails and add up the shapes that you put on it, you get the original picture back. And the shape, he said, that you need to use looks like this. Well, it's a peak with little ripples out to each side. Like if you drop a rock into a pebble into a, you know, still water, it'll go out in circles, the diminishing heights.

So it's kind of like a bell curve except the tails of it get wavy rather than just. Yeah, they're wiggly. Okay. And they're small compared to the major hump in the middle, but they're there. Well, in the real world, we can't have things that go out to infinity in both directions. We have to have things of finite size. So the engineers of the world have said, well, you can replace that hump with the infinity of wiggles

with a hump with one or two wiggles and pretty much have the same effect as the full-blown shape. And the idea, the central idea seems to be that this, on the bed of nails, you're measuring the height of the paper at each of your little. points, but you don't reconstruct it by constructing like a little piecewise constant thing where you just add a value. You don't put a little square there. You don't put a little square there. You put a, you put this shape there and you add up all the shapes. And as long as you don't have

too many high frequency wiggles, you can completely convey the smooth image in a discreet amount of information. That's true. And getting, you know, you put in the caveat, you've got to, in order to honor the theorem, you have to be sure to get rid of the super high frequencies because they'll mess everything up. But people know how to do that.

Yeah. So, I mean, those are super short distances that the eye cannot perceive anyway. So if we lose that, it's not a big deal. That's the idea, right? That you put it off into parts, into noise that don't make humans, uncomfortable. And this is all, I mean, I love it because it's all part of the technical side that most people don't need to know about, but you can, you can capture. It's beautiful. If you, if you can follow the theorem, it is so gorgeous in that mathematical way. This theorem, which is just

completely non-intuitive, works. And we use it. We're using it right now. Everybody uses it all the time is what made the modern world work. Well, and you also help me understand why certain, you know, digitizations of old movies are not very good because they didn't use the sampling theorem correctly. They didn't. They didn't know what they were doing. They, you know, they really did use a little square model. Right. The one thing that annoys me, you know, I watch a lot of TV, and the one thing that continues to annoy me is smooth motions jerk. And Allison, my wife doesn't see it. I'm sitting there going, damn it.

There's a jerk. She says, she didn't see it, right? I said, well, I can't believe you don't see it. To me, they're just awful. These jerks instead of a smooth motion. But it means to me, like the person who's converting from old film format to modern video is not doing the frame conversions correctly according to the sampling theorem. So they're putting in these old.

They're putting in approximations that don't work if somebody sensitive like me is watching. That's what I'm saying. Well, but it's, I think it's kind of reassuring to the person on the street to know that this kind of technical detail helps the conversion from analog to digital be much more faithful than we might have thought. Yeah. It can be, if you're willing to, if you do it right, it can be extremely accurate, extremely accurate. Another place where I'm sort of surprised that people don't object to the lack of appropriate re-construction, sampling and reconstruction is spinning wheels on cars. Do you see them go backwards?

Oh, yeah, sometimes. They go backwards all the time. No, no, they don't. And doesn't that bother you? That's an artifact. Well, does someone like you need to know a lot about how biological, vision work, so the limitations from the eye and from the visual cortex? I thought so. That's why I spent a lot of years learning how the color system works,

color theory, but it turned out not to be all that useful. The closest I got to, you know, I made a contribution, which it's the HSV to RGB color transform. When I started working at Xerox Park with my friend Dick Schaup, his machine had red, green, and blue primaries because that's how video works, red, green, and blue. You adjust the voltages on the three guns, and you get all the colors possible with those three primaries. And I went to him and I said,

Dick, do you know how to convert RGB to hue saturation and value? Because we artists don't think in terms of RGB. we choose a hue from the color circle, then we whiten it with white or we darkened it with black. And he said, well, nobody's ever done that. Why don't you do that? So I did. It was my first contribution. I went home and did it overnight. It was real easy. What amazes me is it still in use. And now it sounds like hue saturation value sounds like how the human perceptual system works. But what I did was just a quick and dirty trick. It's not at all built on what the human eye actually does.

Right, okay. What the human eye does is awesome. I guess you can have a closing thought of your own. My closing thought is I certainly hope that the future of the great digital convergence continues to honor both the technical side and the artistic creative side of things as much as you were pushing it to do for these past few decades. Yeah, I guess what I'm 80, just shy of 82 as we sit here. And I've watched, I was born before computers and I've watched the whole damn thing unwinded.

And it's been awesome. Hasn't it? Yeah. What really angers me is I don't have another 80 years because it's going to be equally awesome. If not, you know, what's it going to be? I want to see. And I don't get to. But there's a lot of prospect out there.

And I think that you set us up very well. Oh, yeah. I think I'm, I'm, I'm so. excited. I'm excited by the art possibilities. One of the first things I did when I started hearing about the mid-journey and those, you know, the picture versions of AI, I called up one of my buddies, David M, who's been making computer art as long as I can remember. He came. and did computer, some of the earliest computer art on the paint program at Xerox Park. And I've known him ever since. And I know some of his early computer renderings have now been used by Herbie Hancock for his album covers. So I call it, I says, what do you think of this AI?

He said, Albi, it's fantastic. He says, you're not threatened by that? He's absolutely not. It says, you know, it's just wrong to think that what artists are, are, are, you know, renderers, draftsman. No, we're not. We're basically, we're the critics for the human race, right? We, that's not his terminology, that's mine, but we, we can say, this is boring, this is trite.

You know, and most of the stuff that these programs turn out is boring and trite, but ever so often, and it's, you know, he uses his skill and taste and education. He says, but look at this. this is fantastic. He'll urge the machine to go off on that direction and explore it some more. Again, it's being used as a lever. It's a tool, yeah. Yeah, which if people like that start saying,

my job is threatened as a creator, then I'll get worried about. I haven't heard any of that yet, yeah. All right, that's good to know. Ali Ray Smith, thanks so much for being on the Mindscape podcast. It's a lot of fun. Bye, bye.