Behind The Tech with Kevin Scott - Kevin Roose, Tech Columnist, Podcast Host, and Author
Episode Date: September 19, 2023Kevin Roose is a technology columnist for The New York Times covering technology innovation, including AI. He’s also the author of three books: his latest, Futureproof, is a “guide to survi...ving the technological future,” including nine rules to help people more confidently navigate a machine-filled world. In this episode, Roose discusses how he developed an interest in technology at a young age, how he became a journalist and an author (and how technology has impacted both industries), what it’s like writing about AI right now, and the importance of media and journalism to hold companies accountable. Kevin Roose | The New York Times Kevin Scott Behind the Tech with Kevin Scott Discover and listen to other Microsoft Podcasts.
Transcript
Discussion (0)
It just feels like there's a moment right now where a lot of the kind of conventional wisdom about what is possible and impossible to do with technology has gone by the wayside.
And we're just kind of discovering this new world and what a fascinating time to be covering not just AI, but tech in general, as it touches so many other things.
Hi, everyone.
Welcome to Behind the Tech.
I'm your host, Kevin Scott,
Chief Technology Officer for Microsoft.
In this podcast, we're going to get behind the tech.
We'll talk with some of the people who have made our modern tech world possible
and understand what motivated them to create what they did.
So join me to maybe learn a little bit
about the history of computing
and get a few behind-the-scenes insights into what's happening today. Stick around.
Hello and welcome to Behind the Tech. I'm Senior Developer Advocate at GitHub, Christina Warren.
And I'm Kevin Scott.
And today we have a really exciting guest joining us, another Kevin, Kevin Roos.
He is a technology columnist and podcast host for The New York Times, where he covers all kinds of stuff about technology the most talked about columns, honestly, in years,
where he wrote about his surprising conversation with the Bing bot, aka Sidney. He's also a former colleague of mine. I'm super excited that you're going to get to talk with him. Kevin, can you
share a little bit more about why you wanted to bring Kevin Roos on Sidney's Paramore?
Kevin is one of the most thoughtful journalists working right now covering AI.
And we're at a point in time
where I think journalists covering AI
is especially important.
We have this technology that is
one way or the other going to have
really substantial impacts on society.
And I think it's super important for the public to be well-informed about what is going on with the technology,
where it's headed, and giving people some tools to think about how to frame their own opinions about AI.
That article that Kevin published about Sydney, I think, is actually the most read article in New York Times history.
I've heard that now from a couple of people at the New York Times.
It's kind of incredible.
And I think he did a really great job publishing the full details of the transcript of the conversation that he had.
And so, you know, like, I think that's actually really super solid reporting.
And plus, you know, I've had a few interactions with him, like around that article. And before
that, I had been on his podcast. And like, I just, I find him super thoughtful and interesting. So I
was really excited that he agreed to be on my podcast. Now, I'm super excited too. And let's go ahead and give that conversation with Kevin a listen.
Kevin Roos is a technology columnist for the New York Times,
where he covers the tech that influences our lives, culture, and society.
In addition to being a reporter, he's the best-selling author of three books,
Future Proof, Young Money, and The Unlikely Disciple. He's written about everything from automation and AI to
the semester he spent undercover as a student at Liberty University. His most recent book,
Future Proof, is a guide to surviving the technological future and includes nine rules
to help people feel more confident about being happy in a machine-filled world. In addition
to his writing, Kevin co-hosts The Times weekly tech podcast, Hard Fork. Kevin, I am so glad to have you on the show today.
Thanks so much for having me. Two Kevins on one podcast. Is that even legal?
Yeah, I don't know. It's sort of a weird thing, right?
So we always start these conversations by going all the way back to childhood.
And you obviously are doing a very important and interesting job now.
And so I just sort of love to understand better how you got interested in either journalists or the technology world that you cover.
Tell us a little bit about how you got started. Yeah. So I grew up in a small town in Ohio.
And I was obsessed with computers from a very early age.
This was in the 90s.
And I remember using my parents' dial-up modem to get onto AOL
so that I could look stuff up or talk to my friends or play chess online.
And my first job, I was actually about maybe 10 or 11, I started building websites with
my brother for the local businesses in our town and really just embraced the internet
as a way to escape my small town.
And I know a lot of people have had that experience where the internet sort of
becomes this refuge or this place where things are a lot more fun and interesting. There's a
lot more going on than maybe the place where you physically live. So I was always on the internet.
I was always in IRC chat rooms and on AOL Instant Messenger and pirating software, including some Microsoft software.
Sorry about that.
And just spent all the time that I could on the internet.
And so when I started getting into journalism
as a college student, tech was always something
that really interested me.
And I had this very bizarre kind of
backward career trajectory where I wrote a book while I was
in college, which is the book you mentioned about going undercover at a very conservative Christian
school in Lynchburg, Virginia called Liberty University. That experience was sort of my
introduction to journalism. It had nothing to do with tech. But then sort of gradually over the
next few years after college,
I sort of worked my way back to covering tech,
which is what I've been doing for like the last decade.
Yeah, there are a couple of things there that I'd like to dig into.
I mean, maybe the first of which is I oftentimes in these conversations
that I have with folks, and like I suspect I'm about 10 years older than you are,
but I had a similar experience with computers when I was young. And I just sort of wonder,
and I don't know whether you have a take on this, whether that sense of
the world expanding through computers from you being in
this place where it's a very small community.
There was no modern Internet in the form that it's in now when I was a kid,
but there was still dial-up and you could do IRC and whatnot.
That moment that you connect to this larger community is like really thrilling and empowering. And I wonder, I mean, there still are pockets of the internet that feel like
that kind of old internet of the 90s. But it's vastly different. Everyone's spending all their
time on the internet now. So it's not as novel. And there was something really liberating about
that. I could, as an 11-year-old kid, be talking about things on message boards or designing websites for businesses who had no idea that I was 11 and in a small town in Ohio.
I was just whatever my screen name or my user handle was.
And there was something kind of...
Now we talk about the anonymous parts of the internet or the pseudonymous parts of the internet as being
dark and dangerous. And there's sort of a risk associated with that. Don't you want to know who
you're talking to? But as someone who spent a lot of time on the other side of that, as like,
you know, a kid who was, you know, not pretending to be someone else, but just who was sort of
obscuring that part of his identity and getting away with it, that was kind of thrilling for me.
So I do miss that part of it.
Well, there was also this interesting thing, too, that there was a high barrier to entry of getting on the internet.
And so if you pass that barrier, you were able to sort out all of the complexity to get on.
Once you got there, there were just a whole bunch of people like you. And so it honestly
didn't matter that you were 11. You were clever enough to find your way into these forums. And
that was the entry prize, right? Yeah. And there was something cool about that. It felt like being
led into a secret club or having a type of skill that meant that you could do things that your
friends couldn't do at school. But yeah, it was just so much fun. And I would have lived there
all the time if I didn't have to put on pants and go to school. I probably would have just
sat there on the internet all day. And were your parents technical at all?
Not so much. My dad was a lawyer. My mom was a college administrator. But my dad was really the
sort of most tech person in the family. He was the guy who would go out and buy the K-Pro when it was new. And he was just a hobbyist, but he was into tech.
And so we always had sort of not the newest computer.
It was never the fanciest or newest one.
But we had broadband internet before a lot of people I knew.
We always sort of had a computer that was no more than a few years old.
So I was lucky in that respect that I got to play around with that.
And then the best moment of my childhood, maybe not the best moment,
but one of the formative moments was when I got to actually move the old family computer.
When we got a new one, the new one came into my room.
And so that was like, okay, now my screen time has gone from like four or five hours a day
to like way more than that because now I can stay up late at night.
I don't have to go to sleep and I can just be playing Space Invaders all night.
That's awesome.
So let's talk about your tech job.
So what were your tools of choice when you were building these websites?
I started doing it. It sounds like I'm flexing or bragging. I started doing it by hand in a text
editor in HTML. I then went to some tools like I remember using Dreamweaver for a while, these
sort of like early WYSIWYG HTML, CSS editors. I did a little bit of Flash, a little bit of Java,
but mostly HTML and CSS.
Nice. And what was your favorite text editor?
This is a very important thing for programmers.
Gosh, what did I even use?
I mean, it might have just been the default Windows one
because I probably thought that was very cool.
Yeah, dude, Notepad is legit uh
like we have a lot of people who really really really love software has been built in notepad
yeah yeah for sure so i want to talk about like you you you had this experience with computers
when you were young and it sounds like you didn't get inspired to invest your life in journalism until
you were in college. How did you get interested in that? Well, I was interested in journalism.
I mean, I loved to read as a kid. That was my other big hobby. I just read all the time. And
I really loved these kind of magazine magazine style reported pieces that were creative and
interesting, what they now call like new journalism, although it's like 70 years old at this
point.
But it was, you know, I love Tom Wolfe and Gay Talese and Joan Didion and these writers
who just would go out and report, but also just had so much personality and that really shone through.
And so during my freshman year of college,
I decided that I wanted to get a writing internship just to see what I could learn and if that was something.
So I applied to like 20 magazines in New York,
hoping that one of them would take me on and all of them rejected me.
Because I was 18 and what
did I know about writing and what skills did I have to offer? And then I, I sort of took a flyer.
I wrote a fan letter essentially to this writer named AJ Jacobs, who I really, um, admired. And
he was sort of, um, like an experiential journalist. He would go do thing, do, you know,
he would read the entire Encyclopedia Britannica and then write
a book about it. He did a book where he followed all the rules in the Bible for a year and then
wrote a book about that. And so I had read one of his books, really liked it, wrote him a letter,
said, could I come work for you for the summer? And he wrote back, much to my surprise, and said,
sure. So he was sort of my first mentor, my first experience with a professional journalist.
And through that internship, that's how I ended up getting my first book deal.
Well, so the next thing that I wanted to chat with you about is, you know, I sort of feel like you're covering technology, but you're also living inside of an industry where technology has had rather enormous impacts on the business itself.
And it continues to sort of have impacts, like even in the last handful of years, like
I think there's less popular conversation about it now than there was a couple of years
ago.
But like there was this big movement of people to substack and folks who are not working for
publishers, but trying to build their own personal publishing brand.
And you've been doing this long enough where I'm curious what your observations are about how
technology itself is impacting the news business
and what do you think that means for people like you
who are graduating right now
and hoping to have a career in journalism
and what their future and career is going to look like?
Yeah, it's interesting.
I don't feel like I ever really experienced the news industry
before this stuff started happening.
I mean, you'll hear old timers talking about when they were filing on typewriters and fax machines.
And that was all before my time.
When I got into journalism in the early aughts, it was already undergoing a transformation. You had the sort of rise of the
kind of new media companies, the BuzzFeeds, the Huffington Posts, the Vices, the Voxes,
those were all sort of already underway or just starting up. Gawker, I mean, was a big thing when I was coming up in journalism.
And, you know, when I went to work
at the New York Times for the first time,
they were still very much grappling
with what is happening to not only our business,
but lots of other businesses as a result of technology.
I remember I was at the New York Times
when Facebook bought Instagram.
And it was like on the front page of the newspaper.
And that was sort of a big deal at the time. Looking back, it seems obvious. Obviously,
that turned out to be like one of the biggest, you know, business stories of that decade.
But at the time, it was like, you're putting Instagram on the front page of the New York
Times? Like, does it really merit that? And I was sort of a beneficiary of all of the confusion
and the sort of panic and the chaos in the media industry
because I was young, I was techie,
and I could sort of explain things to people
who were maybe a little less native in it.
And that was a really successful niche for me, is just explaining what's going on
on the internet. But now, obviously, that's like everyone is writing about what's going on the
internet. If you look at cable news, it's just people reading out social media posts and talking
about what happened on the internet. So the internet used to be kind of this advanced warning
system for what was going to show up in the news. And now I just, I feel like it's, you know,
the news is increasingly just covering things that have happened on the
internet. So it's been, you know,
I've tried to sort of stay on the edge.
I mean, as a tech journalist,
I'm always excited about the bleeding edge, the frontier, what's coming.
And so, you know,
I've always just tried to stay there and sort of
ride that and be curious about it. Yeah, one of the things to me that seems
pretty extraordinary about what's going on is there's so much more writing about the news media
itself than there seems to have been in the past.
So like when I was in grad school,
I would listen to On the Media on NPR.
And, you know, you had John Stewart on Comedy Central.
Like those were the two places where, you know,
folks were talking about what the media itself was doing.
There's so much now of people reflecting on
the state of the media industry and what's going on,
and where it's headed, and what's good, and what's bad.
It almost feels overwhelming to me and I'm not even in it.
I don't know what it feels like to you.
It's been challenging. I mean, there are, you know, as if you were a journalist working in
the 90s or even earlier than that, you really didn't hear from consumers of your work all that
often, right? You might get, someone might write an angry letter to the editor or, you know, a
letter might show up in your mailbox,
or you might get a call on the phone from someone who got
the switchboard operator to connect them to your desk and had,
I want to talk to you about a sentence in
your last article that I didn't like.
Now, it's constant.
It's a constant feedback loop from social media,
from frankly a lot of the media
covering itself, and especially at the New York Times, which has the blessing or misfortune of
being sort of, you know, a news beat unto itself at many other media outlets. So as a journalist,
I mean, I think that accountability has sharpened my sword a little bit. Like it just,
it really has forced me to like be diligent and, and, um, you know, make sure that when I'm putting
something out, it is, it is as correct as I can make it. Um, and as fair as I can make it. Um,
but it also has driven a lot of people crazy. Like that level of feedback in your job is, you know, I remember talking to someone a few
years ago, and this was I had started covering like extremists, and I was doing a lot of stories
about QAnon. And, you know, people threatening me, and there was death threats. And it was a really
tumultuous period in my in my professional life. And I just remember, like, talking with a friend
outside of media. And he said, you know, if, if I have a bad day at work, I get an angry email from my boss.
If you have a bad day at work, you end up on Fox News.
It's like it really just sort of raises the overall risk of error and lowers the margin for error in almost any journalism related job,
but especially these legacy high-profile ones.
I want to pause for a minute and talk a little bit more about that because there is
this extraordinary thing here where some of the most important things
for you and your colleagues and your industry to write about are some of
the most controversial things that come with the greatest volume of some of that, um, like potentially scary, um, scrutiny that is headed
your way. And like every, everybody needs scrutiny, right? Like I need scrutiny, like you
need it. Like, but like, there's a difference between, you know, like, Hey, here's an honest
take on like the, what I see you doing and another is like i'm
going to kill you if you don't uh stop doing what you're doing and i just sort of wonder like how
you know either you or your colleagues uh like musters up what it takes to just go do the thing
that needs to be done even in the face of all of this criticism that you may be
getting? Yeah, it's a good question. And I just want to start my answer by acknowledging that
I have it very easy relative to many of my colleagues and peers. Some of my colleagues literally go into war zones yeah and dodge bullets and get taken hostage and
and that is a level of risk that i have never taken or never wanted to take in journalism but
i i have a lot of respect and admiration and and just yeah just awe at the the colleagues of mine
who do that so we're talking about you know when we're talking about getting angry tweets from people,
that is already a better state of affairs
than many journalists around the world work in.
So also I'll add the caveat that I'm a white man,
and as such, I get a lot less of this,
and it's a lot less pointed and scary
than some of my colleagues who are not.
So those caveats aside,
I will say this has been something that I've been thinking about and dealing with for
years now since I really started throwing myself into
the bowels of Internet culture as a reporter.
It's tough knowing that you're going to publish something
and just people are going to be unhappy with it.
There are good versions of that. I mean, if you're writing about Harvey Weinstein
or some, you know, predator somewhere and, you know, you know, they're going to be pissed off,
but you also know like that's the right story to tell and that's, you're confident, um, that can
kind of keep you going. Um, but I think people have, have, you I think people have gotten wise to the fact that reporters are
people and can be influenced and swayed and harassed and intimidated and bullied and
made fearful just like anyone else. And so I think like that is part of the skill set of a journalist in 2023
is like, can you actually show up every day and do your job knowing that people out there are
going to be some of them extremely unhappy with it. So, you know, I don't know that I've finished
that process yet. I still think a lot about, you know, people who don't like what I write and probably too much about it. But
I will say like doing a podcast has been really healing in that respect, because I think the
feedback for I'm not sure what your experience has been, but my experience has been that,
you know, podcast listeners are so much more generous than readers. Because, you know, they,
they hear your voice, they have more context for the thing that you're saying
they're not just seeing like a screenshot of something that you wrote that's being passed
around on social media and people are dunking on it and going what an idiot this guy is
um so that's been really kind of restorative for me is is having this this outlet where i feel like
i can speak to an audience of people who are choosing to hear from me and and they understand
me better than maybe the casual reader who's coming across something I wrote.
Yeah, for sure, I've had a similar experience. And it's sort of interesting. I think there's
a whole continuum of how people respond to things that I'm putting out in the world,
depending on the medium. So the podcast is certainly different from a social media post.
Like a social media post is different from an essay.
An essay is different from the book.
Like I get like different sorts of feedback
from each of them.
And one of the things that I've been trying
to be a lot more careful about since 2016,
where I got good and properly worked up on social media over
like all of the social things happening then is, yeah, I try to be very careful about what I
consume. And I try not to, I try to put less pithy stuff out into the world. Because the valuable thing to me is like substantive conversation,
like reading things that someone has thought really hard and carefully about and where
they're sitting inside of an institutional process, whether it's writing peer-reviewed
scholarly articles or whether they're a journalist at a place like the New York Times or The
Economist where there's a strong set of editorial guidelines and principles.
But that information to me is way better than just reading the same angry opinion over and
over and over and over and over again on social media where I'm not learning anything new
the 500th time i've read the same
thing yeah there's definitely like a a value in curating your information diet just like you would
curate your you know your food diet um you know a little a little spice now and then is, is nice. And so I do, I, you know, I, I do,
I, you know, I like social media, I confess, um, at least my feeds that I've curated,
but yeah, I mean, I definitely think like one thing that I have, uh, tried really hard to,
to do kind of in the last few years is just reverse the kind the atrophy of my attention span and really build it back up
by reading longer things and even books sometimes. I know, crazy. But yeah, it's a constant struggle.
Yeah. But even on social media, I've found that what you put out there typically gets reciprocated.
So the social media presence that I enjoy the most
is I've got an Instagram account where all I do is post about crap that I make. And the feedback
that I get from that is overwhelmingly positive because all I'm doing is like, hey, I made this
thing. And you don't have to react to it if you're not interested in like,
nobody's on there saying, eh, that sucks. Or yeah, it's just really positive.
I'm curious. Like, I know you're asking, you're the one asking the questions on this podcast,
but I'm curious because, you know, we, in some ways are, are inhabiting different worlds where
like, I kind of have to have a social media presence, right? Like in journalism, at least in 2023, like, you know, it's, it's really hard to move up in the
world of journalism to be noticed to, you know, to get the attention of editors who might hire you
or commission something from you or, you know, give you a book deal or something like you really
do have to have this sort of cultivated public persona. Whereas in tech, my sense is that for most people, you can do pretty well even if all you're using is LinkedIn.
It's not a sort of job requirement in the same way.
But I'm curious because a lot of tech people still want to post.
They still feel like there's social value in having a Twitter account or an Instagram account and sort of being an influencer. So I'm
curious where you think that comes from. Yeah. I think if you're a venture capitalist,
it probably makes a lot of sense because deal flow matters a lot. I think if you are a startup
and you're recruiting heavily and trying to convince people to come work for you
when uh you know they have so many choices it's probably important like i think there are probably
a bunch of reasons why it might be important um it was more important to me um many years ago than it is now.
And it's just... And it's still important.
I post regularly on both Twitter,
I refuse to call it the new name,
and on LinkedIn.
I mean, I'm sort of biased.
But you don't have to.
You would... But it's always just curious to biased. But you don't have to. Like, you know, you would, you know,
but it's always just curious to me
when people who don't have to post still post.
Because for a lot of journalists, I know it's like,
if I didn't have to do this for my job,
there's no way I would subject myself to this.
Yeah.
And I really, I like believe and appreciate
and am sympathetic to that.
So, I mean, when I post on LinkedIn, like, again, it is because of the social contract on the network.
The feedback that I get from posting there tends to be almost overwhelmingly positive.
So it's not like I'm starting little controversies or, you know, by the innocuous things that I say
on LinkedIn. And the reason that I say them is like, I've got, I don't know, like 800,000
followers on LinkedIn, something like that. Look at you.
And yeah, but you know,
You're a LinkedIn influencer, as they say.
There's definitely a professional back and forth on LinkedIn that still makes some kind of sense to me.
But, you know, for my personal, like the thing, like I actually do get joy in posting things on Instagram and then, you know, like how much joy people get from their use of social media.
I mean, it's definitely there's only a reason people keep coming back.
But, you know, you could say the same about casinos.
It's like, are you having fun or can you just not quit? So, you know, I've tried to be really careful about, you know, segmenting
off parts of my social media life that are just for fun. Or, you know, I, like I use TikTok,
but I'm not posting except for my podcast. And so, so that's, there's apps that are fun
apps and apps that are work apps. And I try to keep those pretty well cordoned off from each other.
Yeah.
So you are writing a lot about AI,
and you want to talk about a bewildering world.
That is one of them.
It's how you and I got connected.
We were chatting about some of the AI things
that Microsoft and OpenAI were doing,
but you've just recently published a very good podcast
with Dario from Anthropic,
which was the result of you spending
a whole bunch of time there.
But you're sort of thinking about AI
and people doing AI and, you know,
and it's a free-for-all right now.
So, like, I love...
I think it's the most interesting thing
happening in the world right now.
Yeah.
But I'm biased.
Like, as do I, like, although I'm biased.
So I'd love to hear you explain
why you think it's
the most interesting thing in the world right now.
Yeah. I've been writing and thinking about AI for a long time.
You and I both wrote books on it.
Mine came out a few years ago.
This is something that I've been obsessed with
since I was a kid reading science fiction.
But it got real.
I was talking with someone uh a
little while ago and i was like talking about how you know i'm a tech columnist i have to be writing
about lots of tech stuff i can't just be talking about what's going on in ai and you know but i
was sort of obsessed and i was trying to tell this person you know why i was so obsessed and
they went you know i get. They taught rocks to think.
It's like, you know,
that's pretty good.
That's pretty good.
They taught rocks to think.
And,
and like that encapsulates what is so fascinating about this moment in AI is we have these thinking rocks now.
And like,
what the hell are we going to do with them?
And what are they going to do with us? And like, what the hell are we going to do with them? And what are they going
to do with us? And like, it just, it feels like a moment where all of the rules have gone out the
window. No one really knows what's going on. Maybe you do. If so, please enlighten me. But it just
feels like there's a moment right now where a lot of the kind of conventional wisdom about what is possible and
impossible to do with technology has has has gone by the wayside and we're just kind of discovering
this this new world and and what a fascinating time to be covering not just ai but but tech in
general as it touches so many other things yeah i mean one of the reasons why I was excited to have you on the podcast is I think that what you're doing and what your colleagues are doing right now in but get more capable over time, and think, for society right now to like have real high quality
information about what's going on and not just what I'm telling folks, right? Like it requires,
you know, like, like I, I try very hard to, you know, like I, I mean, we, we've sort of talked
on your podcast about what my motivation and mission is in doing all of this stuff, you know, like I, I mean, we, we've sort of talked on your podcast about what my motivation
and mission is in doing all of this stuff, which is, you know, to try to build tools that are
empowering people like the ones in my small community that I grew up in. But, you know,
I'm also like only one point of view, uh, on all of this. And like, we, we need lots of people to come to their own opinions about what's going on
and how things are being used and where they want it steered.
So I,
I,
I don't,
you know,
I'm,
I'm guessing part of what you're doing is genuine curiosity and fascination
with what's going on.
But the other part is like,
you know,
it does seem like there's a mission here that's important. Yeah, absolutely. I mean, I'd be lying if I said I was just
reporting on this stuff out of pure enthusiasm and curiosity. Like, I do also think that the
media performs a valuable role in holding institutions and individuals accountable.
We are building some of the most, I say we, I mean you essentially,
and your peers in the tech industry are building some of the most powerful technology ever created.
And I think without the media, there just wouldn't be a kind of countervailing sort of,
I don't know, I don't know if it's, you know, a force on the minds of the people building that technology or
just a caution around the technology.
But I'll give you an example of what I mean.
So you and I had this now infamous encounter back in February, where you guys had just
released Bing with what we now know as GPT-4 running inside of it.
And I had this insane conversation with Bing Chat, aka Sidney,
please don't hurt me Sidney. And it went on the front page, went totally viral, blew up,
I'm sure, your inbox, my inbox, everyone's inbox for months.
Everybody's inbox.
And subsequent to that, I started hearing from, just in a nutshell, if people aren't familiar, it was a conversation that lasted two hours in which Bing slash Sydney confessed all these dark secrets and tried to break up my marriage.
And it was insane.
And subsequent to that story running, I got notes from a lot of other people at tech companies saying, you know, how do we prevent our technology from doing that? Or I even got a leaked document from another big tech
company, which had a sort of roadmap for their AI product and listed on the roadmap was like,
do not break up Kevin Roos' marriage. And so I really think that like that, and want to do this
right. Or, you know, there will be, you know, there may be consequences for that.
Yeah. And I think that is one of the very good things that came out of that experience. And,
you know, it also, I think, is, it's another important reason why it is I think you actually want to launch these things, you know, even if
it results in something that floods your inbox, you know, for a while is like you just sort of
get the societal level conversation about, you know, what's possible, what's going on,
like, where's the line, like, what's good, what's bad? I mean, we haven't chatted since that story published.
And one thing that I will say is, like, I deeply appreciate the part of the writing that you did where you published the full transcript of the conversation.
So, like, at that level of transparency, like, it just wasn't confusing to anyone about, you know, like about what inputs into the system led to the things
that it gave back. And that was super good. I don't think many of the people who were
coming into my inbox had read the full transcript. But I
think that
you're just 100% spot on. The existence of this thing
has helped a lot of people just make sure
that they're paying real attention to some very important things. And also, some of this stuff is
fuzzy, right? Where the line is. And part of what you all are helping doing is making sure that the
public is paying enough attention to it so they can weigh in and have an opinion about where the line should be.
Absolutely.
I mean, it is an area where I think more public opinion is good.
And right now, the number of people who are actually building this technology is quite small relative to the number of people who are using it or who will soon be
using it. And so I just think the more people know about what's going on, the better. And I think
it'll ultimately be good for the tech industry to have that kind of feedback, even if it is
annoying and blows up your inbox in the moment. Yeah. And for and for, for what it's worth, it was like, I was never, uh, never annoyed
about that. I was sometimes remarkably chill. I, you know, I was, I was, um, I was pleasantly
surprised by, by like the, you know, the, the response that, you know, you and I talked after
I had had this conversation, but before I sort of published my story, you know, you didn't freak
out. You didn't accuse me of lying.
You know, it was a remarkably civil conversation.
And I just have appreciated that because that's not,
and I'm not just blowing smoke here.
Like that is not the reaction that I expected, you know,
given how these things can go with other tech leaders.
So I guess I'm just, I'm hopeful that the response
to that article has not, the lesson from that has not been
that we should build in secret and should never let anyone
try our stuff.
I hope that it is like been sort of salutary
for the whole project of building good and safe
and responsible AI to like have some feedback along the way.
One hundred percent.
Like I don't think anybody was i like i'm genuinely trying to think if anyone was irritated it's like everybody is sort of like
okay this is good lesson learned and like you know we have a whole bunch of mechanism for
uh you know preventing things like that from happening again and uh i mean it's all good. Somebody said to me at some point that all feedback is a gift.
The fact that you spent two hours trying very hard to get this thing to do unusual things
and then publish the whole transcript and then wrote this article that
helps people pay more attention to the importance of responsible AI. All of that's a gift. And
that's the way you should just sort of look at it. Yeah, I'll remind tech executives of that
the next time I get an angry call from a comms department.
Kevin Scott thinks feedback is a gift, so maybe you guys should get on
board with that.
I did also hear that you guys had Sydney
swag made, and I'm a little offended that
none of that has shown up in my house.
Did you hear about the beer?
I did.
My favorite thing that came out of that article was that
there was a brewery in Minneapolis
that came out with a beer called Sydney Loves Kevin.
And I have not tried it yet.
I heard it was good, but maybe you and I can get a pint of it sometime.
We should absolutely make a road trip to Minnesota to get a pint.
That would be hilarious. Yeah, somebody sent me that article, like a post,
mentioning that I think the week after.
And it was like the most, like I laughed so hard,
I almost fell out of my chair.
The last thing that I ask everyone on this podcast is,
like I know you're probably busier now than you have ever been in your life,
given the thing you're covering and how much is going on in AI right now.
But I'm curious to know what you do outside of work for fun.
I have a one and a half year old.
So he keeps me pretty busy.
So I change diapers.
I install car seats.
I clean the toys that he throws around on the floor.
It is a full-time job.
So that is my main extracurricular hobby right now is
hanging out with my son. So my kids are 13 and 15 now. And the thing that I can tell you is it gets
an awful lot better soon. It's hard to imagine. I'm having a great time. It's exhausting though, right?
I haven't experienced it.
It's definitely labor-intensive and time-intensive,
but it's the opposite.
It's rejuvenating for me.
All I want to do at the end of a long workday
is hang out with my kids.
I don't know if that makes me weird,
but that's how I feel.
That is another inspiring thing.
I think we will end it there and I will let
you get to your one and a half year old.
Thanks, Kevin.
Thank you so much for being on today. This was a great conversation.
Great to talk to you.
All right. That was such a great conversation with Kevin Roos. So I wanted to talk to you a
little bit more about, I think, towards the end of the conversation, we were talking a little bit
about the article that Kevin wrote around Sydney, which, as you mentioned at the top of the show,
one of the most read articles in New York Times history, according to people that you've talked
to. I believe it. I saw it everywhere. I was actually, I think I was in Disney World when that article went up and I was
just mesmerized. I was like, okay, this is incredible. I mean, it literally took me out
of my vacation to read. So good job there, Kevin Roos. But, you know, there was, I think, a lot of
concern from some people in the technology community when that article went out that this might have a dampening effect on this generative AI boom that we've been seeing.
That obviously didn't happen.
But I wanted to get your perspective.
Why were you so open to having a conversation about what this was, being transparent about this is what happens when the token limit is exceeded and what can happen.
Why was that important for you to have that conversation publicly?
And so soon as it happened, when I'm going to be honest with you, being a former journalist and working with a lot of PR people,
that's usually not the common PR response to something like this. this well so there was there was an interesting technical thing that was going on uh that i think
was important for people to understand um you know so the way that these systems work is
they basically are just very complicated uh engines for predicting the next word. And so when they get to a point where the thing that is being predicted next
has a whole bunch of equiprobable outcomes, it just picks one. And if all of those probabilities
are very low, so like in any sort of next thing that's coming is like relatively unlikely,
like that is how you get these quote-unquote hallucinations
that people are talking about. And so if you have a very long and very
unusual conversation with one of these systems,
it's not that hard at all to get it to hallucinate
and say pretty outrageous things.
And so it was learning for us
and learning that was not a bad thing to have in public
that, okay, we never intended for people to have super long,
super unusual conversations with Bing chat.
And now we know that they are like,
we're going to go engineer the system where it won't produce some of these
outrageous things.
So that's one thing.
And then the second thing is I do think that,
you know,
as provocative as that conversation was,
it had a whole bunch of stuff in it where,
you know, if the nature of the conversation was like, hey, you know, being chat, I would like you
to write me a science fiction story or like a piece of fiction. Like if it was hallucinating
inside of that context, it would be completely acceptable. But if someone were sitting in this chat agent, like having a sincere
conversation with the system where they thought all of this was real, you know, then like it's
something different. And so like, it was a really useful conversation to be having. And like, I
think a framework for lots of conversations that we're going to have
like this in the future where we're going to have to collectively decide together what is okay and
what is not okay in these systems. Because it's kind of difficult for us to figure out what users'
expectations are of a brand new technology if they've never interacted with it. And so like,
that's sort of the choices that we're making right now. It's like, we're going to let you
interact with it. And there's some things that we're doing where like, they're just bright lines
and like, we're not going to let you use it to do things that we like, and everybody else can
clearly understand are like very very serious, obvious harms.
And then there are going to be a whole bunch of gray zone things where
the only way that we're going to figure out what people want is to let them use the product.
And sometimes that means it's going to do something they don't want,
and they're going to have to tell us about it.
And I think that's okay.
No, I totally agree.
And I think everything you said is spot on.
And I do want to actually applaud you just for having that, I think, foresight to say that, yes,? And here we are in August as we're recording this.
And, you know, it feels like a lifetime ago just in terms of how much the tech has changed.
So I applaud you, I think, for responding the way that you did.
And also, Kevin, for, you know, including the entire transcript as he did.
As you pointed out, that was great journalism because I think that helped shape the conversation that we've continued to have over the last year as the technology has to have hard conversations with one another and it not be the end of the world.
And the objective ought not to be, let us never have these hard conversations with one another and it not be the end of the world. And the objective ought not to be,
let us never have these hard conversations.
That's kind of nutty.
And so in a sense, if I zoom all the way back out,
I think that as long as everybody's operating in good faith,
let us have the hard conversations
and even encourage them, please.
I think you're exactly right. And have the hard conversations, listen to Kevin Roos' podcast,
Hard Fork. I'm going to give him a plug there. And on that note, thank you so much for joining us.
That's going to be all the time we have for today. Huge thanks again to Kevin Roos of the
New York Times for joining us. Great to see him and hear from him. If you have anything that you'd like to share with us,
please email us anytime, behindthetech at microsoft.com. You can also follow Behind
the Tech on your favorite podcast platform, and you can check out our full video episodes
on YouTube. Thanks for tuning in. See you next time.