TED Talks Daily - How AI is decoding ancient scrolls | Julian Schilliger and Youssef Nader

Episode Date: January 24, 2025

AI researcher Youssef Nader and digital archaeologist Julian Schilliger share how they used AI to virtually "unroll" and decode the Herculaneum scrolls, burnt and buried by the eruption of Mount Vesuvius nearly 2,000 years ago. Learn how AI could help decipher a range of artifacts, revealing clues about the mysteries and achievements of the ancient world. Hosted on Acast. See acast.com/privacy for more information.

Mayday, Mayday. We've been compromised. The pilot's a hitman. I knew you were gonna be fun. Now playing. Tell me how to fly this thing. Mark Wahlberg. You know what the difference between me and you is? You're gonna die. Don't shoot him! We need him! Y'all need a pilot?

Flight Risk. Support for the show comes from Airbnb. I have traveled to some amazing places this year, and one of the highlights was definitely Nepal. The energy was electric and the community and kindness unparalleled. I love staying in Airbnbs when I travel, but there are also lots of reasons to host on Airbnb as well, like some extra income and putting your home to good use while you're away. And if you travel a lot like me, it makes much more sense than letting your home to good use while you're away. And if you travel a lot like me, it makes much more sense than letting your home just sit empty. You could be earning some money to go

towards your next trip. I'm excited to think about hosting as a flexible fit to my lifestyle. Your home might be worth more than you think. Find out how much at airbnb.ca. You're listening to TED Talks Daily, where we bring you new ideas to spark your curiosity every day. I'm your host, Ilyce Hwu. Today's speakers did something rather remarkable. They used advanced AI and computer vision to virtually unroll ancient scrolls buried long ago by the eruption of Mount Vesuvius. Machine learning researcher Yousef Mohammed Nedder and digital archaeologist Julian Schilliger shed light on their approach

and explain how AI can open up new possibilities for understanding ancient knowledge. We always think about the potential of AI changing the future, but what about the potential of AI changing the past? My name is Yousef Nader. I'm an Egyptian AI researcher and a PhD student at the Free University in Berlin. And last year, I led the Vesuvius Grand Prize winning team on exploring this very question. You see, the story starts almost 2,000 years ago.

A Greek philosopher that we believe was Philodemus of Gadara sat in one of the many rooms of the Villa de Papyri. He talked about music, he talked about pleasure, he talked about what makes things enjoyable, questions that still plague us until today. One of his scribes wrote down his thoughts on sheets of papyrus. The sheets were rolled and stowed away

for later generations. Fast forward 150 years, not even more, Mount Vesuvius erupts, burying Herculaneum, the villa, and the words of the philosopher under a sea of hot mud and ashes. Now fast forward again to the 17th century. People are excavating around the area. and ashes. Now fast forward again to the 17th century. People are excavating around the area. They found beautiful statues, breathtaking frescoes, and some weird looking pieces of

charcoal. This is when the first scrolls were discovered, and people were racing to excavate more of these. What knowledge is included that is not known to us now? What things should we know about these scrolls? My name is Julian, and I am a digital archeologist. When the pyroclastic flow hit the scrolls, it had a destructive effect. It tore into them shred-off pieces, and it charred them badly.

People 250-something years ago were curious what's lying inside those scrolls, hidden and not accessible anymore. Because of a lack of technology, they had to resort to physically unrolling and thereby destroying most of the scrolls. To this day, only the most damaged and deformed scrolls remain in their initial rolled up configuration. Fast forwarding a little bit, the computer age arrives,

Youssef and I are born. We are going on and getting our education. And, well, at the same time, Brent Seals, a researcher and professor, had the idea to use CT scan technology to actually digitalize the same time, Brent Seals, a researcher and professor, had the idea to use CT scan technology to actually digitalize the scrolls with the hope of one day digitally unrolling them. It's a difficult question how to unroll this digitally. Nat Friedman, a Silicon Valley investor, also saw this research, and he wanted to help.

That was in 2022. He reached out, and together with Brent Seals, they created the Vesuvius Challenge, with the goal to motivate nerds all over the world to solve this problem. They created a grand prize promising eternal glory and monetary incentives to anyone who could do that. I myself saw that in the internet while writing my master thesis at ETH Zurich in robotics and I was instantly happy to solve it.

At least try, you know. And I went on, joined the Discord community where all the people that were also contestants and playing with the scroll data were exchanging ideas. I joined there and started working on it. Also, there on Discord, I met Yousef and Luke, who would become my teammates and with whom I would actually win the grand prize. Surprisingly, it went on and made global headline news.

It even got into the British tabloids. So, when we started, there were two main problems still remaining. One, you had to unroll the scroll, and two, you then had to make the ink visible. Yousef will tell you more about that part. For me, the most exciting thing was the computer vision problem

of unrolling those scrolls virtually. I decided to iterate on a tool that was created by the Kentucky researchers and make it faster, less prone to errors, and just iterate on it and make it better. The Vesuvius Challenge team saw that and also implemented a team of ten people that would use my tool. They would annotate scroll data where they created a red line where the surface would

lay. The algorithm then would take it into 3D space, creating a three-dimensional representation of the surface. Computer algorithms would then flatten it and create a segment. This all would be called segmentation in the space of the scrolling and unrolling community. So I created open source commits to this tool and implemented new algorithms from my studies

like optical flow to better track the sheets through the volume. First off, those were really small segments. And I added improvement, made the code faster, and had lots of feedback from the community. They were really happy, and I was happy getting lots of feedback. It was a really positive environment.

So in the end, I could track the performance of the algorithms, how the segmentation team performed, and I could see that my improvements from start to finish would be around about a 10,000-fold improvement over the initial version. This algorithm was then also used to unroll all the area that you can see in our submission. All the sheets were generated with these methods.

In December, I was looking for teammates. I made a blog post and I showcased my newest algorithms, reaching out to anyone that was willing to team up. Yousef and Luke got into contact with me. They were happy to team up and I was happy as well. So after the virtual unwrapping, the words still are not visible. The main problem is that the ink that was used at the time was a carbon-based ink, and carbon-based ink on carbon-based papyrus in a CT scan isn't visible,

or at least to the naked eye. So the same team at the University of Kentucky decided to test whether the ink is present at all in the CT scans. For this, they took some of the pieces that, you know, people broke off, the scrolls, and they fed them into the same pipeline of the X-ray CT scanning, and this gives us the 3D data that we're working with.

You can also, because you can see the ink and it's like an exposed surface, you can even improve it with infrared imaging, and this gives you a ground truth of what letters you're actually trying to find. And then from there, you can train a machine learning model to try to find these letters. The way this works is that the model looks at very small cubes at a single time and tries

to decide whether there is ink present in this area or not. And then when you keep moving this cube all around, the model gets to see different data samples and then tries to understand what ink actually is. So this is how it looks while the model is training. The model is starting to see the letters perfectly. So the data is there. The ink is there, but it's just very hard to find and see. There's a different way that you can find this ink, one

that actually scales very, very well. So this is where my journey begins with the Vesuvius challenge. There is this neat idea in computer vision literature where if you don't actually have labels, if you don't have the goal that you want your AI model to reach, you can pick an intermediary goal along the way.

So one idea to just let it know about the structures and familiarize it with the data is to tell it, show it different views of the same image and tell it that these are the same images. And after that, you take this model and you train it like the previous models that the University of Kentucky did. And while the approach doesn't fully work, it also doesn't fully not work. And the first image that was produced by the model, there were some very faint signal in there. It seemed like the model is catching on something, but it wasn't clear exactly what the model is catching on. So I decided to take this prediction and create a new ground

truth, asking the model, hey, I think these might be letters, I think there's something in there, try to find more of this. And my ground truth actually has four correct letters and four other delusions, but that was okay. So training a new model with this data, the model started to find more ink, find more letters, and the lines even looked complete. So I thought, what are the chances

that if I do this again, the models keep improving? And this was the core behind our grand prize winning solution. Repeating this process over and over, the models kept improving. The main trick was you needed to prevent the models from memorizing what the previous models have learned. You're essentially asking the model to learn what the other model has learned So over fitting was a serious problem that required a lot of experiments But in the end got getting the recipe right we were able to predict

All of these letters without the models ever seeing them these were the first ten letters This was the first coherent word read from an unopened papyrus sheet. From there, scaling the process, within weeks we had now columns of text, even special characters that the papyrologists found very interesting that the model was able to find. The approach was open sourced and the data

and the code was out there and the race for the grand prize was on. Recovering four paragraphs at an 85% clarity. And the key to our success was perfecting the data and the model with so many iterations and so many experiments. In the end, we were able to recover more than 14 columns of text and 2,000 letters. Two thousand characters safely started away two millennia ago. In just nine months, we

In just nine months, we discovered them again. AI helped us in large portions, writing better code and even being part of our algorithms. It opened a window into the past. What's next? Let's open this window more. AI will help us access information that was so far safely locked away. In the words of the author, we do not refrain from questioning nor understanding. And may it be evident to say true things as they appear. That was Yousef Mohammed Nedr and Julian Schilliger at TED AI Vienna in 2024. If you're curious about TED's curation, find out more at ted.com slash curation guidelines.

And that's it for today. Ted Talks Daily is part of the Ted Audio Collective. This episode was produced and edited by our team, Martha Estefanos, Oliver Friedman, Brian Green, Autumn Thompson, and Alejandra Salazar. It was mixed by Christopher Faisy-Bogan. Additional support from Emma Taubner and Daniela Balarezo. I'm Elise Hue. I'll be back tomorrow with a fresh idea for your feet.

Thanks for listening. I knew you were gonna be fun. Now playing... Tell me how to fly this thing. Mark Wahlberg. You know what the difference between me and you is? You're gonna die. Don't shoot him! We need him! Y'all need a pilot?