99% Invisible - 507- Search and Ye Might Find

Episode Date: September 14, 2022

Adam Rogers has been thinking and writing about what’s known in the industry simply as "search." For the last decade, people have been grumbling about not being able to find things online, both in our private data and on the public web, despite ever-evolving algorithms. Ever since humans started writing stuff down, the struggle has been in how to organize it all so that its contents wouldn't be lost in the stacks. Search has always been an attempt to fix that problem.Search and Ye Might Find 

This is 99% invisible. I'm Roman Mars. This is going to sound like a stand-up comedy bit, but I was on my way to the airport recently, and I needed the confirmation email from my flight so I could get my boarding pass. So I pulled up my phone and I typed in some keywords into the search bar of the mail program. I started with the airline I thought I was taking, but that search pulled up mailing list promotions and old flights, and I know if I search long enough and hard enough I could eventually find it, but the thing is, by now I'm at the airport, and I just have this naive conviction that searching should be better and easier.

But I often come up empty. There are days when I feel like I've control effed up my life into oblivion and I'll never easily find the email I need, much less old photos and documents on my hard drive. The good news is, I'm not alone. Adam Rogers feels this particular pain too. You know, you think emails are problems, so try to find a slack you once wrote. You know, try to find it tweet, try to search for a photo on Flickr or Instagram, Pinterest. Like I dare you. Adam's a senior tech correspondent at Insider.

He's been thinking and writing about what's known in the industry as search. For the last decade, people have been grumbling about not being able to find things online, both in our private data and on the public web. I would say to people, I think search sucks and other people would say, oh my God, it really does. For Adam, this takes for this go beyond me not being able to find that one email.

I will tell you, I'll tell you a personal story. It's one of the ways that I started to think about this. My grandmother was a professor of library science at UC Berkeley. When Adam's grandmother was alive, she lived in a small apartment. Every wall was filled with rows and rows of books. She was a librarian. They were really well organized. The books held all the knowledge that Adam's grandmother wanted to access.

It was arranged by topic and author complete with important search tools. Notes and tabs stuck into all the various volumes that she could reference when looking to pull up some tidbit of information. Adam was charmed by the whole thing. But as she got older and was a little bit less coherent, she didn't remember stuff as well. And the notes became less and less coherent. And I find that terrifying. Adam's grandmother had built a working search function, and it failed. All the same millions of pages of information were still there, mining the walls, but she

couldn't access this archive of knowledge. According to Adam, the same thing could be happening when we search for anything on our computers. More and more, we outsource our information, our personal information, and our autobiographical memories. We outsource that more and more to digital media now. Not just the sum total of human knowledge, but also our own photos, emails, and memories stored in the cloud. As we put more and more of that somewhere else besides our own heads, search becomes more and more of a existential crisis, not just an annoying thing where I can't find the email with my confirmation number in it.

We begin to potentially literally forget ourselves. Our ability to search and retrieve information at our whim feels like one of the most important developments of the digital age. So how do we get to a point where it feels like search is failing us? And how do we fix it before it's too late? Today the word search sort of feels synonymous with the word Google, but I think it's actually one of the oldest design problems in the world. From the time humans started writing stuff down,

the struggle has been how to organize it all so that its contents wouldn't be lost in the stacks. Search has always been an attempt to fix that problem. I think for our purposes here, it does make a lot of sense to think of search as a designed experience. It wasn't just walking into a library and wandering around. Ever since, you know, the priest who ran the libraries at Alexandria knew which scrolls went in which cubby hole or however they did it.

And leading up through the DeSple system or library of congress numbers, these things are search. These things are what makes search possible. Long before we could search for things online, Google was essentially a person, a reference librarian. If you wanted to find something on say, growing vegetables, you could go to the gardening or farming sections of the library, but in the thousands of books in that huge section,

you'd quickly get overwhelmed. That's where reference librarians and archivists come in. They take your topic and help you narrow it down even further, applying their own nuanced knowledge and specialized training to help you search better and find exactly what you're looking for. That's how search operated for centuries by topic mediated by human to human interaction. And it worked pretty well. Good evening, Dr. Bush. Well, I good evening, Ed. But eventually, this easy flow of search hit a snag.

In the mid-1940s, the snag was highlighted by an American engineer named Venever Bush. Well, sir, I've known you a number of years, and last week, I managed to mispronounce Vannever and I apologize. Well you've got plenty of company there Ed most everyone must pronounce it. I think the record was held by one postmaster who managed to pronounce it from four different ways and of course one of you. Dr. Bush. At the time Vannever was convinced that problems with search were

hindering human progress. He pointed to Mendel's ideas on genetics, for example. These ideas were lost for an entire generation, he said, because they were buried in the avalanche of newer research. There was no efficient way for people to parse through all that information. But Vennyver was a big believer in the potential of machines. The analytical machine, which will supplement a man's thinking methods, which will think for it, will have his great effect on his grasp of the world and his access to daughter and so on, his manipulation of it.

He will have his great effect in that way, as the invention of the machine way back took the load off of men by giving them mechanical power instead of the power of their muscles. So in 1945, Venevier took to the page and dreamed up an imaginary futuristic solution to the problem of search, a machine called Memex. The Memex would make search easier. It would look like a desk. There'd be a keyboard, viewing screens, and storage space for all of human knowledge, as long as it was on microfilm and could fit into a dust drawer.

On the left side there would be all the information in the universe, and it would all have links, and then on the right side, you would follow those links for the information you wanted. So the search became about connections within what you were looking for. Theoretically, the user could teach the mimics which words were relevant to each other. So if the word culture in one document makes me think of death, I could tell the mimics to connect those two words. Then, when I searched for the word culture, all the documents featuring death that I previously linked

would show up. I could scroll through all the results by turning a crank. In essence, the mimics user could build their own little analog algorithm for search. The mimics was never built, but Paul Con, someone who wrote a whole book about many of his mimics. Well, in the 1990s, he animated how the Memex might have worked. First, he runs through an encyclopedia, finds an interesting but sketchy article, and leaves it projected.

Next, in a history, he finds another pertinent item and ties the two together. Vinny Verkold this process of linking two keywords together, trail-building. The idea of searching documents not by broad topic, but specific word and then linking related documents together turned out to be right on the mark. About 50 years after Vinnyvar dreamed up the mimics, his ideas about search came to fruition with a little thing called the World Wide Web. a wide web. This mimics-inspired idea of searching by keyword became the new default. We're at searching by topic like you would in a library, is similar to looking through the table of contents of a book. Keyword search is like using the index. It is much more precise, and searching by keyword worked well for a time when you type in a word and get

only a couple of dozen results. But as the digital information space gets bigger and bigger, things begin to evolve and change. In 1993, there were 130, yes, you heard that right, 130 websites on the internet. Three years later, there were over 100,000. So that starts to happen very soon when the World Wide Web becomes something that's more than just an academic,

more than just the thing that connects all of the physics labs. Doing an online search by keyword was now posing problems because the internet was starting to feel crowded and clunky and the usual ways of searching were feeling outdated. The kind of searches that I might have done on Lexus Nexus are pro quest in that 80s or 90s. These carefully constructed searches

were you had to pay per search so you really wanted that one search to work because you couldn't keep doing it because it would cost money. And that felt like a very constructed and constrained experience which brings us to when search, as we know it today, was born. A couple of kids at Stanford come up with another approach to searching, called PageRank, and that becomes the basis for Google.

Google runs using PageRank. And what PageRank says is, we're going to count And what PageRank says is, we're going to count how many things link to something. The idea behind the PageRank algorithm is not only will your search engine find the site's feature in whatever word you asked for, but it will find the best sites with that word. We're going to say that if something has a lot of links to it, that's a better source of information. That we now prioritize that above things that have the same kind of information in them, but don't have as many links. As the Internet expanded, more and more spammy or irrelevant web pages would come up in search results. PageRank helped filter those out by emphasizing high quality results. A .edu link or anything from the BBC, for example, would rank higher

in a search result than say in the blog. In part, because many other sites would be linking to the more well-known pages. To me, this is super cool. I love this as an insight, because it turns out that that's the same way that, like, memories are reinforced in the brain. How many times do we go back to them? It's the same way that ants tell each other, there's food over there, but not over there. If an ant walks over the trail and leaves pheromones on it again and again and again, that trail becomes more important to the colony. Pretty soon with the help of PageRank, Google became a verb.

And since everybody goes to them, they have statistics on what people search for and how they search for them. So they have this incredible statistical oomph. You them, they have statistics on what people search for and how they search for them So they have this incredible statistical oomph You know, they get billions and billions and billions of searches every day and only about 15% of the searches that they see in a given day Are new that they've never seen before so 85% of the searches that the world does on Google every day are things they've already seen They've seen that search before and they can reproduce it type in in some words, and here's the stuff. Here's access to human knowledge. Google's the biggest, bestest, most innovative website.

Well, I guess search engine in the galaxy. Here's a clip from a TV segment in the early 2000s offering its viewers tips on how to search on Google, the hot news search engine in town. So we're here to pump you up with a few tricks that will bring that power back to your browser. There are seven tricks in total. The first one really easy to do will open up Google here in Mozilla.

By the way, they just released a new version of Mozilla at Mozilla.org. They're up to 1.2 now, very, very good browser. I've got a number of things. With Google at our fingertips and the proliferation of search bars on every digital interface, it felt like we had finally made it to the very top of human knowledge. And it felt like not a constrained experience. It felt like, oh, that's done. That's fixed. It works.

In fact, the Google search bar with all of its millions of data points is so good, it changed our expectations of what search is. And today, that's part of the problem. We were all trained very well to think, oh, well, now search bars are just like the Google search bar everywhere. And everywhere I see a search bar, it's going to be just as good as a Google search bar is. And then you try that on Amazon. For many of us, when we type a query into an e-commerce website, we expect that the results

will be ranked for us by relevance to our search, but that is not how it works. So a place that's trying to sell something is trying to sell, like if it has more of one thing in its warehouse than another, it'll try to push that on to you. If it has something that's on sale, it might show you that first. If it has a product where the people who make it have a pay for play deal with the e-commerce site, it'll show you that stuff first. The result is that the thing you search for that you are trying to buy will be buried by results for stuff that the company wants you to buy. You can put in the specific name of

something in the search bar in Amazon and it won't show up until you scroll to your three screens down. And that, I mean, I'm still shocked by that. I'm naive or I'm stupid or something. I'm stunned like, no, I put in the name of the book. I know what book I want. And you still got to scroll to find it because there are all kinds of other competitive commercial pressures.

The same thing happens with private search when we're trying to retrieve something like a flight confirmation number or a vacation photo from the cloud. On some level, we expect every search bar to work like Google's, even though it isn't Google. So personal information management turns out to still be kind of an unsolved problem more so than wide search. Adam says that the reason searching your own files isn't as streamlined as web search is because your personal email or photo database is private. Software engineers can't improve on algorithms as easily as Google can on the web because with their private data, there's less statistical commonalities to draw from.

Great assault here. Google supposedly doesn't release information about this, so we don't know for sure about all that. What we do know is that not being able to find our stuff can have consequences. And that's a personal and also a cultural amnesia that I think becomes troubling. You lose moments from the past. You end up only existing in the moment like where you share the stuff and then it doesn't exist anymore because you can't go back and search it.

In the years since Google rose to prominence, it set the standard for what search could be, but lately that glow seems to be fading. And in fact, it turns out that almost as soon as it was born, Google almost without us noticing began to fail. I found it incredibly chilling to be talking to somebody who works on search at Google and have that person say, yeah, search really isn't a solved problem. And it hasn't been for a long time. In the early 2000s,

Google started getting bogged down by monetization and people trying to gain the system. One way this played out was with SEO or search engine optimization. SEO is basically the process of getting a web page ranked higher in web search results. In the early years of SEO, webmasters would often stuff their web pages with keywords to get them to rank higher, regardless of actual relevance, and the strategies keep evolving. I was just reading some folks today talking about this on social media about how like everything you look for on Google has these like 2,000 word introductions of meaningless gibberish, like before you get to the recipe,

or before you get to the instructions for changing the memory card or whatever. And that's because one of the things that Google came to prioritize, it seems, they're not that open about these things, is how long somebody spends on the page At all that up and this is what you get when you type in a web search query It can feel like you're playing hide and seek but for information to find the thing you search for First you'll need to scroll past rows of stuff labeled ads or maps and horizontal boxes of questions

You didn't ask. Another reason search engines like Google so often miss their mark is because, unlike say, a librarians approach, which might be more like, here are 10 bucks you could read to try to figure out on your own. Google tries to give you the most popular quote-unquote best search results. In other words, a direct answer. It can sort of understand what's on a web page, find the information that it thinks you are looking for based on its statistical analysis of all of the billions and billions of searches

that it sees all the time, and feed you an answer. And we now have come to think, oh, well, that must be the answer, then. But when a search engine prioritizes the visibility of one seemingly popular answer over another, sometimes it can lead to misleading or even harmful outcomes, like in 2021, when the result of a single Google search sparked an uproar in India. An angry flashpoint has emerged in Karnataka after search engine Google showed Kanada language in bad light calling it the ugliest language in the country.

Huge outcry was triggered after this news. Someone had typed the words ugliest language in India into Google and the search engine embarrassingly answered. Google named an actual language in India and insulted 43 million people in the process. Doutrage prompted Google to issue a detailed statement apologizing for the gap and said they would improve their algorithm. But the Karnataka government has made it clear and apology is not enough.

They could send a legal notice. So in line of all this, search engines are evolving again. To fix the issue of private search, not working like the Google search bar, some search companies are supposedly considering creating search engines that could dig through both your personal information and go out on the web. You'll pay us a little bit of money, and then it'll be private. And we'll search your personal information as well as the web, and we'll do it really well, you know, so you'll be able to find your flight confirmation number

and your freaking flyer number, and the name of that person who you worked with that one time, and all that other stuff that we searched for in our own computer, that one document. And to fix the issue of constrained single answer results, Google announced last year an AI technology that would attempt to understand not just what the searcher is typing, but what the searcher is thinking.

Meaning, the search would no longer be driven by topic or keyword, but by what the AI thinks the searcher meant by those words. You can have a kind of iterative back and forth, because you'll have a search engine that actually understands language instead of just looks for keywords. There's a promise in this, and there's also a risk that as search companies start to behave more like humans with their own ideas, they might continue to algorithmically nudge us away from our intended searches and into unexpected directions, which feels creepy and consequential if you think about all the life decisions that you make in a given year based on information you process after googling.

But Google's vision of the future points to something already starting to happen in the world of search, this time on the part of the searcher. One move that a lot of people now use is they will use Google with the keywords that they have to search for, but tell it to only search Reddit. with the keywords that they have to search for, but tell it to only search Reddit. Increasingly, people are turning to Reddit to search for information, but they're using Google to do it, because Reddit's own search function is supposedly not very good.

So you can use Google to find the subject, and then you have human beings actually answering your question. On Reddit, someone looking for information on keeping their garden vegetables alive can ease drop on a conversation between niche experts in humid or desert climates for a particular plant. Or they can engage in a real human conversation about say how much water does a thing actually need using Google to search Reddit for human conversations. It's a little janky, but it may be the best

solution for a search that we have right now. In a way, it comes close to replicating the experience of talking to a good old-fashioned librarian. The thing that the conversation with the reference librarian that was the classic model of search would give you is if you didn't really know what you were looking for, the reference librarian would help you figure out what you were looking for.

It turns out you were looking for something different. It turns out there's a whole related thing. There's a whole other room of the library that has more that you might want. Between Google's new AI and the wild world of Reddit, the future of search is beginning to look a lot like the past. Ultimately, the questions pertaining to search aren't getting easy answers anytime soon. Questions like, could there be a better way to order the world's information? How do we organize the stuff that we know so that the next person can know it too?

And what time am I flying out of JFK anyway? These are some big questions, the answers to which we may never know. Coming up after the break, I talk with our digital director, Kurt Coleset, about his personal favorite approach to searching. So when we were making this episode, Losh and I were light in part on another 99PI colleague to help us better understand search, both its history and work around to use search more effectively. And so we called on Kurt Colestead Digital Director. There's a reason why he's called Digital Director. And so he's joined me to go through

both of those in a little bit more in depth. I've been steeped in this world of search for a very long time now. And like everybody else, I have to search for stuff, but I also ran a number of web publications which relied in part on visitors from search engines for ad revenue.

So yeah, it's fair to say I have spent some time studying SEO. Yeah, and that is search engine optimization, which I know kind of peripherally from having stuff on the web for a while. Yeah, so what is that? Well, as a person who ran websites using SEO, what is that all about? So a lot of it is white hat, right?

It's like above board stuff like, hey, I've got a local architecture firm and when people search for art and architecture firm in this area, I should show up in the search results. But a lot of it is ethically much murkier, and it's been that way basically from the beginning. Okay, so what are some examples of the beginning of people using SEO in a bad way? Right, so I have one of me using it in a bad way,

which is that, yeah, I'm not a teenager. I call it coming from inside the house. That was coming from, um, you know, call it's coming from inside the house. Yeah, it's coming from, yes, yes. So in the, in the 90s, I built a fan page for a band that I will never tell you the name of, um, out of sheer embarrassment. And one of the things I did was add in any word I thought somebody might search in relation to a band like other similar bands and stuff like that.

Just tons and tons of these words at the bottom of the page in small format. And sort of half hidden, you know, with the color that kind of matched the page. I mean, I was, you know, basically, yeah. Well, turns out this is a well-known tactic called keyword stuffing. And it used to kind of work.

Like Google used to be a little bit simpler and would just kind of read what's on the page and try to deliver results. But of course, these days, the AI is much more sophisticated. And if you actually try to do that today, Google will not only catch you, but they will like potentially delist your site for doing something like that.

Hmm, okay. So if the algorithm is getting smarter and catching these kinds of things, why isn't search just getting better and better? Yeah. So the search engine optimization industry is huge and they're highly motivated. So every time Google comes up with something, they come up with something else. And it makes sense if you think about it. Like they throw so much money at this because if they can get their business to come up first in a search organically, as it's called,

like without having to pay for ads or anything, it can mean the difference between success and failure, right? So everyone is forced to get more clever about their strategies and then Google for its part has become a lot less transparent over time about what they factor into their search results. And for good reason, right, to deter these unethical players who learn the rules and then learn how to cheat from them. Yeah. So it's kind of this arms race, but also like things get more secret as part of the arms race as well.

And so, you know, given the current state quantitative things, you know, what is your advice for people trying to search and find stuff? Well, it depends upon the kind of thing as you'd imagine, but I want to walk you and our listeners through my personal favorite strategy, which honestly, I use to search for everything from beard trimmer reviews to tips for taking care of suckiness. Okay. And so, let's start with the first example. And that's called a transactional search, right?

Like, I'm looking to buy a beard trimmer. Okay. I mean, I know that you mentioned it. I seem to need a new beard trimmer all the time, so I'm apparently not buying the right beard trimmer. So, so. Same here. They always seem to break.

Yeah, they do. Allmer. So, so. Seeming they always seem to break. Yeah, they do. All right. Yeah. OK. So I could start on Amazon and type in beard trimmer and then sort those results by average customer reviews because that way at least hopefully you get the best ones first.

Yeah. And that can work. Or I can search in Google and click the shopping tab. And, you know, I'll get some results there. But here's what I actually do. In Google, type in something like Best Beard Trimmer, then site colon reddit.com. And that way, you'll only get results from Reddit's website. And for those who don't know, Reddit is this huge website with tons of subforms called

subreddits. And there are some of these that are really big like news and some are really niche subjects like Ben's hair care. And I'm pretty much, you know, I follow with the logic of this is, but I guess my question is, why would you trust Reddit, over say like reviews on Amazon or third party website? It seems like it's still user reviews in a way. Yeah, I mean, that's a great question. Because you wouldn't think, it just seems kind of random, right?

But the reality is that review websites, particularly ones that have gained their way to the top of Google, are a crapshoot. You might find legitimately good and helpful reviews, or you might find somebody who's writing about a product specifically because they'll get a commission if you click a link on their site and buy that product. And then on Amazon, reviews have become sketchier

and sketchier over the years for various reasons that some of which we talked about in this episode. Yeah, and you can feel it. You know what I'm saying? When you look at Amazon. Yeah, you really can, right? But on Reddit, there are these subreddits for virtually anything and gaming every related Community to a certain product or or something

It would actually be pretty hard to you. Maybe someday AI is will figure it out But for now, it's kind of out of reach for most SEOs And with that beard trimmer search, for instance, I put this in and I landed on conversations in a subreddit called OneBag, which is a community I didn't even know about, that's all about minimalist travel. So that's cool, right? That's relevant. Like those guys probably know what they're doing.

And then I also found a search result in Buy It for life, which as you can imagine, is people discussing high quality things that hopefully will last the rest of their lives, right? Not cheap beer trimmers like ours to keep breaking. And the key part to this is there's no financial incentive for these people, right? Like they're not linking you out to products with commissions. They're just communities discussing these things openly, and they might not always be right, but there is this voting system. And so good things tend to rise to the top there.

So if I were to do a search like this, would I want to look for the highest voted result? Is that how it works? Yeah, I mean, that is a good place to start at least. And beyond that, too, usually there will be responses to that top voted thing. And beyond that too, usually there will be responses to that top-voted thing. And people will discuss things back and forth pointing out downsides or suggesting alternatives in these comet threads.

Wow. I mean, I think I've noticed this happening more and more that Reddit has been part of my searching for information. And it's really kind of makes a ton of sense. And so that's very cool. But I never thought of it as something so practical as looking makes a ton of sense. And so that's very cool. But I never thought of it as something so practical as looking for a beer trimmer. It was always like a piece of information or a few people discussing, yeah. Totally, totally.

And that's the sort of other domain of search, right? Is this category that's called informational search. And Reddit is really good for that too. So again, sort of starting from the beginning here. It's like you could go to YouTube and look for a tutorial about how to take care of your succulents. And often you'll get a pretty good results. But if you're just watching one video, you're getting one person's opinion and review websites, they're also usually monetized. So there could be a conflicting reason that they're making all these, making all these videos, right?

They might not really be experts. And, you know, that obviously can work against your need to find impartial and useful advice. Right. So I assume this could take us back to Reddit where we could actually like some evaluation of even these videos. Yeah, exactly. And it's basically the same process as before, right? You search for a few keywords, you limit your search by putting site colon reddit.com. And I would argue that as good as reddit can be for product reviews, it's even better for expert advice because there are so many these tiny,

niche subreddits about everything. So take succulents, for instance, there's a huge gardening subreddit, a smaller indoor gardening one, and an even smaller one that's just for succulents, and for some types of succulents, they even have their own whole community, which to me is crazy, but kind of beautiful too. Yeah, so this all makes sense to me, because you're really relying on human knowledge and voting and all the types of things that we've counted on since Web 2.0 came about.

But my question is, you know, you can type a search into search bar on Reddit. So where are you using Google to search Reddit? That's a great question, too. I mean, I've tried that, believe me, I've tried that a lot of times, but Reddit's built in search is, in my opinion, a mess.

And it does offer various ways to search. So it kind of looks sophisticated, like you can search within a single subreddit, and you can sort results by, you know, the age of the post or the vote count on it. But ultimately, searchability has never really been the company's focus, and it shows when you actually try to search the site. So by using Google, you're combining this really powerful search engine with this really remarkable wealth of personal knowledge. You're cutting out all the bad results of random review websites, and you're just going

straight to the site that often has good information. And of course, it won't work for everything, but it's a starting point for so many of my searches these days. And I should point out that the sort of third major category of search is something that Google actually does do very well. And that's called a navigational search. Basically, if you know what you're looking for, you're looking for a particular website or a person, plugging that into Google is often a great way to find them, right? Yeah. I mean, this is what we all do. But for these informational and transactional queries, I do suggest starting with Google plus Reddit and then just going from there. Wow. This is great. This is like real news you can use stuff right here.

Yeah. So I'm going to give the hybrid search a shot and I'll let you know how it goes. Excellent. Yeah, I mean, yeah, rarely are we so practical at our, in our episodes. But yeah, hopefully people find this useful. I should mention one other thing before we go. And that's that for some people, Reddit will set off a bit of a red flag because the site is a whole, is a very mixed bag. It's sort of like its own version of the worldwide web with warts and all. So for people who have a negative

association with Reddit going in, I hear you and you're right, there's a lot of toxicity on that website. But if you search hard enough and avoid the bad neighborhoods, you can find great little communities too. Well, I do. I have noticed the words. Yes. I also have noticed really nice communities. Yeah. Our subreddit is beautiful. I must say. Exactly. I like the, I like to, hang out in the blankies subreddit. And also just like the front pages really good. I'm, you know, that's an n people are funny. You'll find a lot of good stuff on there too. But it's duly noted, like, let, you know, just be careful out there, have your wits about you, and try to find the best information possible. So, right, well, thank you so much, Kurt. This is great. Yeah, anytime, Roman.

99% of his vote was produced this week by La Shaman Dawn, edited by Kelly Prime and executive produced by Delaney Hall, original music by Swan Rial. Sound mixed by Amidik Anatra, fact checking by Liz Boyd. Kurt Colstad is our digital director, Thorey Satim includes Vivian Le, Jason D'Leon, Chris Barube, Christopher Johnson, Emmett Fitzgerald, Martin Gonzales, Jacob Moltenata Medina, Joe Rosenberg, Sophia Klatsker, and me Roman Mars. Such a thanks this week to Adam Rogers for being Adam Rogers and a warm welcome this week to our Fall Intern Olivia Green.

We are part of the Stitcher and Serious Exam Podcast family, now headquartered six blocks north in the Pandora building. part of the Stitcher and Serious XM podcast family. Now headquartered six blocks north in the Pandora building. In beautiful, uptown, Oakland, California. You can find the show and join discussions about the show on Facebook. You can tweet at me at Roman Mars in the show at 99PI org.

We're on Instagram and Reddit too. You can find links to other Stitcher shows I love, as well as every past episode of 99PI at 99PI. dot org. I love as well as every past episode of 99BI at 99bi.org.