In Our Time - Cryptography

Episode Date: January 29, 2004

Melvyn Bragg and guests discuss the origins and history of codes. In October 1586, in the forbidding hall of Fotheringhay Castle, Mary Queen of Scots was on trial for her life. Accused of treason and denied legal representation, she sat alone in the shadow of a vast and empty throne belonging to her absent cousin and arch rival Elizabeth I of England. Walsingham, Elizabeth’s Principal Secretary, had already arrested and executed Mary’s fellow conspirators, her only hope lay in the code she had used in all her letters concerning the plot. If her cipher remained unbroken she might yet be saved. Not for the first time the life of an individual and the course of history depended on the arcane art of Cryptography.What are the origins of this secretive science? And what links the ‘Caesar Cipher’ with the complex algorithms which underpin so much of our modern age?With Simon Singh, science writer and author of The Code Book: The Secret History of Codes and Code-Breaking; Professor Fred Piper, Director of the Information Security Group at Royal Holloway, University of London and co-author of Cryptography: A Very Short Introduction; Lisa Jardine, Professor of Renaissance Studies at Queen Mary, University of London and author of Ingenious Pursuits.

This BBC podcast is supported by ads outside the UK. What makes people want to believe in aliens? I'm Tristan Redmond, one of the hosts of the Global Story podcast from the BBC. Donald Trump last week announced that he'd be releasing the US government's UFO files. So why the renewed interest in life out there? And what deeper spiritual meaning might people be searching for? Check out the global story. We are serious journalists on BBC.com.

or wherever you get your pods. Thanks for downloading the In Our Time podcast. For more details about In Our Time and for our terms of use, please go to BBC.co.com.uk forward slash radio four. I hope you enjoy the program. Hello. In October 1586 in the Forbidding Hall of Fotheringay Castle, Mary Queen of Scots was on trial for her life. Accused of treason and denied legal representation,

she sat alone in the shadow of a vast and empty throne, belonging to her absent cousin, an arch rival, Elizabeth I of England. Walsingham, Elizabeth's principal secretary, had already arrested and executed Mary's fellow conspirators. Her only hope lay in the code she'd used in all her letters concerning the plot. If her cipher remained unbroken, she might yet be saved. Not for the first time, the life of an individual and the course of history dependent on the arcane art of cryptography. What are the origins of this secretive science, and what links

the Caesar cipher with the complex algorithms which underpin so much of our modern age. With me to discuss the history of cryptography are Simon Singh, science writer and author of the code book, The Secret History of Codes and Codebreaking, Lisa Jardine, Professor of Renaissance Studies at Queen Mary, and Professor Fred Piper, Director of the Department of Information Security at Royal Holloway University of London. Simon Singh, can you tell us how secret messages were communicated before the inventions of the first forms of cryptography. What were the early examples of what we call steganography? And what is that? Yeah, no,

steganography is something that people don't talk about much nowadays, but it's, what you do is, well, with cryptography, you hide the message by scrambling it up so nobody can read it. It's there in front of you staring in the face, but nobody knows what it means. With stegonography, you hide the existence of the message

itself. So, what the Chinese did was write on silk, wrap the silk into a wax ball, wrap it in wax, sorry, swallow the ball and then the messenger goes on his way. Any guard would, you know, check the messenger but not find any paper or anything to see.

Another nice example which I think the Greeks used was to shave the messenger's head, tattoo the message on the scalp, wait for the hair to regrow, send the messenger on his way and at the other end you shave the head and reveal the message. So this is an invisible

inks, all of these techniques where you cannot see the message. So we're talking about the Babylon, the Egyptians, the Greeks all using these techniques. Yeah, and everyone has their own... And people are continually being inventive. So one example, again, is instead of writing on a wax tablet, you scrape off the wax, write on the wooden board, put the wax back, and it looks like a blank or an innocent letter, but at the other end, you scrape off the wax and you find the message. So you're continually trying to innovate in case somebody

discovers your original steganographic technique. Another message was to... Another way, it was to kill the messenger, wasn't it, so that he couldn't give the message to anybody else? You had this disposable messenger system. Yeah, no, I think that's the equivalent of chopping down somebody's telephone line today, is you just, yeah, chop the messenger. And that obtained for a very long time. Now, that was the method. How did it change from hiding the message literally to hiding the message in language?

We obviously talking about an increase in literacy, for one thing, but are we talking about increased sophistication? Can you just take us from that big stage, I'd switch from steganography to cryptography. Yeah, I think, I mean, one of the interesting things is that the Chinese worked with steganography a lot because with Chinese characters, there's not much you can do to them.

A whole word is a character, a whole concept is a character, and you can't jumble it up. But once you have a Roman alphabet or a more conventional alphabet, then it's much easier to either take the letters and jumble them round, to make a kind of anagram, that's transposition, or to substitute the letters for different.

different letters or symbols. And so that's where the transition happens between steganography, where you're hiding the existence of the message, and cryptography where you're hiding its meaning by, as I say, jumbling the letters or substituting them. So that's fascinating, that sort of coding goes hand in hand with the alphabet. We decode speech one way and we encode it with the same methods. Absolutely. Can you tell us, have you any idea when cryptography came in? I think as soon as people start writing down information, people immediately start writing down secrets, whether it's personal diaries or whether it's political or military strategies and messages.

As soon as people write down secrets, they need to think of mechanisms for hiding those secrets. So I think there's an example of a Babylonian potter who comes up with a recipe of a glaze and wants to protect that recipe, and so it's been encrypted. And the remarkable thing about that is that somebody had to decipher cunei form, which to us seems like a code because we didn't know what it meant. And then once you've unraveled the true meaning of how cunei form works, you then have to unravel a code that somebody's encoded in cuneiform. So some remarkable achievements in decoding codes.

Yeah, reading up on this, one of the things that impresses you is the sort of intensity of that particular sort of intelligence that went into the whole operation. Fred Piper, one of the earliest substitution ciphers that we know, know about, as well written about Caesar, who did two ciphers, can you tell us first of all about the Caesar cipher, and a rather dramatic occasion where it comes to our attention, and now

we can talk about the Caesar shift cipher. Okay, the first Caesar cipher is, basically... This date is about, we're talking about in the middle of the... You tell us. Yeah. The Gallic Wars. Yeah. The gulac wars. And the first idea of Caesar appears they have was just changed the alphabet. Middle of the first century BC, yeah. So change the alphabet.

Instead of writing in Roman letters, just substitute for each Roman letter, a Greek letter. send it in Greek letters with Roman meaning, people won't understand it, and you've got your secret message. And all you need to know to decode is exactly what's happened. But people were just unaware that this type of thing was going on. And he did this rather dramatically, as he describes when the Gaelic was, to raise a siege which was being held by Cicero.

By Cicero, and the courier was a little nervous of crossing the line. So the courier threw the message on a spear to tell Cicero. This road the hope was coming, but the spear got stuck in a tree. And for a couple of days, the troops were just waiting, not knowing what's happening. The message telling they were saved was above them. I think it was the third day that eventually they realized it was there and read it. Now, the Caesar shift cipher is rather different. Here, what you're doing is replacing the letters of the alphabet by other letters.

And the Caesar cipher simply explained is just shift every letter, three letters to the right in the alphabet. So A would become D, B becomes C, and then at the end, X goes around to the front, it becomes A, Y becomes B, and Z becomes C. And so now what you see is not what the message says. It's what's called a coded version or a cryptogram. Now that seems to be quite simple, but it held for an amazing amount of time, didn't it? Yes. Why is that? Well, I can only assume...

When I say amazing amount of time, I'm talking 900 years that was still operating. Well, in its simple form, probably not quite for that long. But the idea of the shift, the shifting cipher. Well, they extended a little to what's called a substitution cipher. So instead of just shifting, because then there are only 26 shifts you can do, they then just made arbitrary substitution. So we could say let A become Q, B become T, and we could choose what the shift was. Not the shift, sorry, what the substitution was.

And now, instead of having 26 keys, you've got millions and millions of. a case. It is true because, sorry, I missed that stage out, entirely my fault, you're quiet. Once you start the substitution, though, but still using the basic Caesar method, you go from having a very, very few possibilities to having something, the figure I've got here

is, once you start the shifting, you've been talking about, our key will be, that A will be excellent. Are you talking about something in nature of 400 million, billion ways? Is it something? It's true, isn't it? Four hundred million billion ways. Forward 26 knots at the end is.

Yeah. To actually play with those 26 Yes, that's right It's astonishing It's a heck of a lot And nobody's going to guess What you've done They'd have to work it out

So no wonder it'd held for 900 Oh yes That held for a long time Yes Then The It was the Arabs So can you just describe

Us Lisa Because you can range Across these things Better than most When the Arabs When the Arabs took up Various what we can call Greek through Roman

Forms of Learning They heightened interest in linguistics and mathematics and statistics for religious purposes. So we talk to a massive example of the law of unexpected consequences here. So what did they do? Well, they studied, they're studying the Quran, as indeed they still would, in meticulous detail. And the Jews were actually studying Hebrew scripture in the same way. And you then begin to see, if you're looking for God's word.

Yeah, but just to be accurate here, it's the Arabs. Let's stick with the Arabs. The Al-Kindi, the Arab philosopher, who was the man who did what we're about to talk about. Absolutely. Yeah. I think I'm suggesting that they might have done it

elsewhere as well. But, so what Al-Kindi recognized was, and this had come out of, I'm sure, out of Mullah's looking closely at the, at the Quran,

was that the frequency, that they're looking for patterns and they identify that various letters occur with greater frequency than other letters when you're in any

written text. So in English, it would be E is the most frequent letter that we Let's go back one step, because it's very important to how this came about, because it came about a wonderful accident of history, because they're not looking to crack codes. They're actually trying to find...

So, and you please come in on this, because it's very important to get this clear. It's wonderful how these things go by indirections. They're looking, as I understand it, they're looking to find the accuracy of the time scale of the revelations of Muhammad when he said certain things, in which order or what time, therefore what weight do they have? Are we right so far? And they're trying to place in time, these scholars,

where these different, when these different relations came. So they're analysing the texts to do with time and they're bringing, and we're doing mathematics not to bear it. And out of that comes this frequency analysis, cryptanalysis. Yeah, it's very sophisticated analysis of text in the same way that we might look at a text now and say, well, did Shakespeare really like this?

it's just the way Shakespeare's structured his language. People would look at these texts and say, well, is this really the way the Prophet Muhammad wrote things down? And they would look at the length of the words he would use, the complication of sentences and clauses and so on. And by studying that for the first time, they begin to realize subtleties like vowels are incredibly common when you write things back.

So it is about, you're quite right, it's about trying to identify authenticity. But authenticity, and it's done right across written language, but it is in this strictly Islamic context that it's first recognized. Whilst you're trying to identify authenticity, you note repetitions and repeated uses, and that then gives you an actual grid

of the occurrence of letters, which suddenly makes it clear that if you've got a straight substitution code where however you scramble the letters, you substitute a single letter, for another single letter, then by looking at a piece of writing of any length, if it were in English,

you would see that there were X's everywhere, and that would be an E. Yeah, can we just go even further into that, Fred Piper? So we're talking about these scholars working in Basra and Baghdad in the 9th century, and this, we haven't time to go into why they got so advanced in mathematics and linguistics and statistics,

but they did. Please describe it. I would like to be described again, because Alkin, well, just because it's so, so important, I think. Alkini said it out very plain. He said, you take a text and then what it is to say? Okay. Well, can we translate into English? Yeah, let's do it in English.

Well, we've got the alphabet that connects with. If you take a text in English and then... A page of writing. A page of writing, just with no particular bias, so no particular words, then E will represent roughly 12% of that page. And T will represent another 9%, and then A and O. and these frequencies are well documented. Now, if all you do is encode by making a letter-for-letter substitution,

then if E represents 12% of the, let's call it the plain text, the clear message, then whatever represents E will represent 12% of the cryptogram or the coded message. And so all you need to do is do a frequency analysis of the coded message, and with different letters it will agree with the frequency analysis of the real message, And so you've worked out what the substitution was. And Al-Kindi did this with Arabic first, and then he used the words that the most frequently we will call the first, and then we will call the second, and then we'll call the third.

And they'll make mistakes with the P's and the Q's and that, because they're a bit not used all that. And also, statistics are never totally reliable. I mean, sometimes, A would be more popular for T for people's style. But given this frequency analysis, it is now possible, I mean, I do a lot of teaching to 13-year-olds, and you give them a text, and they can break it, just by nothing more than giving the table of the frequencies in English,

and let them count and do it. Now, so we've got cryptography, the doing of it, and cryptanalysis, the undoing of it, and we're in the 9th century, and so the stage is set for the sort of battle of codes for the last 12, 11, 10, 12. Have we done enough now to move on to the next? I'd like to say one thing because you picked up about growth of literacy.

Actually, it isn't so much growth of literature. Well, it is growth of literacy, but when messages are passed, illiterate interceptors are on many occasions quite capable of passing it to a literate person. But what is, I think, important, is empire. Because it's the reach of empire that requires communication to the furthest regions. You might not be able to get one messenger all the way. and in fact the boundaries of an empire are generally defined with the Roman Empire or with the Ottoman Empire

or with the Habsburg Empire by how far you can get provisions, armaments and message and communication. Absolutely. So cryptography escalates and I think that's what we're watching as we move from the 9th century on. The Roman Empire you see an absolute point

where unlike the Greeks we have this reach and therefore the need for communication to reach the edges. Now, as we move forward into the Renaissance with competing empires, we will get this sense. You have to write it down. Ideally, you wouldn't write a message down, but if the empire is big enough, you have to write it down. Simon.

And just the, I mean, the frequency analysis that's just been described, sounds very simple and straightforward. You know, you look at the text, look for the most common letter, that must be E, the second most common letter must be T and so on. But there are some lovely subtleties about frequency analysis. other properties of language such that vowels are very sociable letters. The letter O will be next to B as in bottom or next to C as in cog or D as in dog. So the vowel will sit next to all the other letters very happily.

Whereas consonants, you don't find a D coming after B or a D coming after G or a D coming after H. So the consonants cluster around vowels, but the vowels cluster around everything. So if I've got a coded message and I see a letter there, which is very sociable and is always sprinkled amongst all the other letters, I can say, well, I know it's frequency and I know it's a vowel. I've got to move on. The example that I started with at the beginning of the programme was the Mary Queen of Cots. Mary Queen of Scots.

Yeah, I know, I know, I know, I shouldn't have an anaesthetic two days before a programme. Mary Queen of Scots. Trial, 1586, Queen Elizabeth's cousin, nevertheless, a threat, because Queen Elizabeth's assassination plots, Catholic Protestants, very worrying for Queen Elizabeth, if you executed a queen, what did that do to the divine right, and really difficult. But she'd been caught up in a plot,

and ciphers had been used. Now then, Simon, can you briefly tell us how that case is said to have turned on ciphers? We persuaded that it was to, and she was convicted, because the cipher was broken, It was a very difficult cipher. The cipher was broken by Walsingham and his guys, and she was convicted on the basis of that.

Right. I mean, the details are slightly murky because you're never quite sure what really happened and what was sort of set up for historians to later discover. But one version of events is that Mary is communicating with plotters who are trying to release Mary from prison and assassinate Elizabeth. Babbington and those guys. And they use a code, because these are clearly treacherous attempts

on the life of a queen. And the code they use is not very different from the one we've just described. They're going to swap A for a diamond. They're going to swap B for a square, C for a cross. They throw in some nice details. They throw in some red herrings, some spurious characters, which just pepper the cipher, which would confuse a potential code breaker. They throw in a character which doubles the previous one.

So if you're doing bottle, you put something after the T to double the T. These messages are sent to and fro where Mary replies back and says, yes, you know, let's go for it. I'm on board with this plot. Unfortunately, the person delivering the messages is a double agent. Giffin. That's it. So whenever Gifford gets a message, he takes it to Elizabeth Spymaster Walsingham. Walsingham makes a duplicate and hands it to the code breaker.

And despite all these little embroideries around the code, you just count the letters, you see which one's most common, you reckon that's E, and you break the code. and there in front of you, Elizabeth Spymaster sees a letter which proves undoubtedly that Mary is attempting, involved in a plot, to assassinate Elizabeth. And as you said at the beginning, with proper caution as a historian, is that is perhaps what we were, it was laid down for us to believe later. There are doubters, Lisa Turde. Well, I mean, the first thing to recognise with that is that it wouldn't be Mary who encoded her own message, so that you're not talking about someone's handwriting,

and indeed, since you're using symbols, you couldn't. So the person called a secretary, who in a secret, a maker of secrets, in this period it's spelled secret O-R-I-E, so it's your man who puts your stuff into secret form. The secretary puts it into code. And so that in fact, Elizabeth needed concrete proof. She was so reluctant to execute her cousin.

needed a bit of paper. So what we don't know is whether these codes that were supposedly put into the bungs of barrels and the barrels were smuggled into a fathering castle and then somebody took the bung and so on and so forth. How are we taking the bung comes from? I don't know. Simon. No, I know. I'll interrupt you. Well, we do. Yeah, we will, we will, you'll tell people next week. No, right. So now, of course we don't know whether this was a setup or if it was real. The fact was she'd been in prison for 18 years.

18 years she'd been in prison. She'd been exchanging ciphered letters all the time because the next thing from history you have to understand is that if the post is leaky as between a castle and the outside world, you will use simple ciphers even if you're just asking for a laundry list. Fred, Piper, where are we with ciphers at this? time in Elizabeth Nage. We're getting the great intellectuals, extremely

interested in it. Francis Bacon is a scholar right across the board, and a lot of people, still thinking of Shakespeare, I'm a lot of rubbish, but so there you are, and he is fascinated by cybers, he construct ciphers, so can you just how is it developing? Liza said, well, sorry, Simon said, well, it wasn't

unlike the Caesar shift cipher. Now, is it developing in any way? Can you just bring us up to speed on that? Yes, I mean, whether whether, you know, Mary Queen of Scott's ciph was broken or not, I wouldn't dream vacuum as historians. The fact is that it could have been broken. I mean, it did rely on the frequency analysis.

Still the same 9th century stuff? For this particular cipher, yes, with a few editions that Simon's spoken about. The truth, however, is that if you want to have a code, in those days what you had to realize you had to do was break these frequencies. And there were various techniques for doing this. One was, I'd say, putting in spurious characters that Simon spoke about, which, you know, if you're a charactering that has no meaning, anybody doing the frequency analysis for the text

will have that as a frequency somewhere, and that throws off their calculation. The other option was to have two letters representing one, and that throws the statistics as well. But over 100 years before this, Leon Alberti had had the idea of, well, let's use two substitution ciphers for one message, and use, for the first letter, use one of the ciphers. For the second one, use the second, then for the third one, use the first one

again. So he suggested using two in rotation. And this, of course, does have the effect of changing the statistics, because now in your ciphertext or your cryptogram, there will be two letters representing E, and two lepros representing T. So it's a lower frequency. So it's 12%, E will be 6%, like lots of other T's and essence. Whatever. So whatever happens, the frequency changes. And this idea was, what, mid-15th century? So it took them 500 years. Mr. Cottonon, any reply to what had been done in Basra and Baghdad in the 9th century? In that particular case, there were lots of people, even the Mary Queen's Sots-Syphe, but Cotton-on, they were trying to destroy the statistics by adding spurious characters and so on,

but this concept of using more than one cipher and the same message appears to have started with Alberti. Now, he didn't really push it until another hundred years later when it came the Vigionaire Cipher. and the Viginaire Seifor then takes the very simple Caesar cipher. Can I just hold on to the visionary cipher for one moment? I want to go to that and it's very important because it actually in a sense takes us to the enigma. But before I just want to go into the, for a few minutes of Eccan releasing myself, so I'm in you, Fred, into the way that you particularly talk about in the Civil War. Cyphus came in, maybe we just, there's more of it,

the populations are getting bigger, literacy is growing, and so. But you do get these people. I've talked about bacon, but there's Wilkins and the... There's Wallace and as Hook. Of course you've written by Hook. Newton's using cipher for alchemy, peeps is using cipher's diet, and codes, and away they're going. Now, it's almost like a rush of cipher's to the head, isn't it? Can you say first?

Now I come to you, Simon. I think, as you were saying to Fred that there's this big gap, the fact is difficulty in encryption responds to need. If you can get by with simple encryption, a lot of code is just about deterring the casual reason. when you have a lot of mail being passed around for all kinds of purposes, some of which are confidential, what you need is that the casual glance doesn't tell you what it means. Now, you can use the Caesar code for that quite happily. You can use very simple codes. And the harder a code, the more complicated the encryption, the harder it is to decode,

even if you are yourself, if you have the key. So we move into a period, in periods of danger, we move to more complicated ciphers. the English Civil War, we suddenly get barrages of mathematicians working on cipher, because you now have life or death, as it was for Mary Queen of Scots, actually, life and death, you know, if Charles I is sending his mail, his instructions to his army, he's sending it across his own country, in his own language, through hostile territory, the code has to hold. And in fact, Charles the first code, which was seized in an ambush,

a whole lot of letters seized enough to get the cut to decode by John Wallace, who remained one of the greatest mathematicians of the Civil War and Restoration Period. And when the kink Charles II came back, John Wallace became master cryptographer for Charles II. Great mathematicians become the people who make code more difficult. But as they make it more difficult, ordinary people like us can't decode. I can decode a Caesar code and a simple double substitution code.

But after that, I put my hands up and say, know somebody more expert has to do it. Simon, so what is that? Can you just develop what these as a set? Yeah, I think it's a case of, is this just my personal diary, which I just want to keep for my own, and I just don't want my servants to read what I'm writing, so that's a nice simple code, or is this a matter of state, could this lead to a calamity of this code is broken? So at the other end of the scale, you've got people like Cardano,

creating a piece of card with little holes poked in it. And so I send my letter, but if you put the card over the letter, you can only see maybe seven or eight key letters, which might spell out the location of where an attack might happen. Is that still steganography? Yeah, it's a form of stegonography, yes, absolutely. And what it involves is having to make this card, cut out the holes, make sure it fits, write the text so it works,

make a copy of that card, get that card to the other end so that they can decode it. And that's hard work, it's complicated, but it's worth it if you've got a valuable message. Fred, now can you take us to the visionary code, which is a subject, can you just tell us about that when it came in and how much of an advance it was? Well, the simple substitution, the Caesar cipher,

which we can all break, right? We can all break the Caesar cipher. The Vigionaire idea is, well, let's have, there are really 26 different keys, right? You shift by 1, 2, 3, 4, up to 26, and then shifting by 26 is the same as shifting by naught because you've gone right around. Now, you've got to explain this to people.

I've got it, I haven't got it with me, it's down here in these notes, but I think I can remember it. You've got a square, and the top line, says A to Z. And down the left-hand side it says A to Z. Oh, right. And down the right-hand side, it says it says Z to A. Yes. And at the bottom line, it says Z to A. So you go, and inside that, there are 26 versions across and down about it. So you've got that square of a multiple, the alphabet. And so A, A to Z, first column, B to A, second column, C to X, and the way it goes.

So the first row is the Caesar Cyphle with Shift 0. The second row is the Caesar Cyph with Shift 1. the third row is Caesar ciphers, shift two and so on, then the bottom row is the Caesar cipher with shift 25. So you've got billions of options now. And now, what, the Vigionaire cipher, you just agree on a sequence of numbers that say, 1, 3, 7. Then you use the first row first, third row second, seventh row, third,

then you come back and use the first row again. So now you're using three ciphers. And if you choose the sequence of four numbers, you'll be using four ciphers, and you use the sequence of five numbers you use. five different cycles, and you keep cycling round and round and round until you get to the end of the message. Right. Now, what does this give that nothing else has given so far? Well, it breaks the frequency. It was invented at the end of the 16th century, but it wasn't really taken up until the 18th century.

Was this because it's too hard? Well, no, it was actually, it's very unfortunate for Mary Queen of Scots, because the publication of the Viginesse cipher, I think, was the year after. I think that's right. The year after she was actually, I mean, it was very close. the effect it has is just breaking the letter frequency and the longer your sequence so the more letters if you like in the cipher text will represent the same letter E

and so the frequency statistics get flatter and flatter and flatter and in theory if you use the sequence that's as long as the message which is where we'll come in a minute then there'll be no information there at all and the breaking of the Visioner Cipher was because they used short sequences and so you kept repeating and it was the repetition

that was then exploited to break it. There's a copy of this on our website. Do you want to say more about the Visioner Cipher, Sam? Oh, must do you. It's a really hard one to explain, but the key thing... Well, we think side to start again. There's a bag to town here, Roche. You've got a square of paper.

Yeah. We all know the alphabet. I think, well, here's one way to explain it. If I want to encrypt the word dog using simple Caesar. Let's, I'm going to shift every letter by one. So D becomes E, O becomes P, and G becomes H. So D-P-H is my code.

If you want to crack that code, I think, well, God, it's a Caesar cipher. Maybe I'll shift it by eight places. No, it doesn't work. Shift it by seven places, it doesn't work. 23 doesn't work. And eventually, once you've checked all 25, you think, ah, it's just a simple shift of one place. So all you have to do is try 25 different shifts, and you'll get it.

With a visionary sipher, you're saying, okay, the first letter I'm going to shift by eight places, The second letter I'm going to shift by nine places. The third letter I'm going to shift by two places. The fourth letter, I'm going to shift back to whatever the first number I thought of was. So suddenly there are 26 cubed different ways. What you're trying to do is break up the patterns. Language has rich, rich patterns, which is how we sort of started.

We said, you know, there are word lengths, vowel relationships and so on. What you're trying to do is get from a very rich patterned message to something that looks completely random. has no patterns in it, which has no structure. Code breakers really devour structure and patterns in order to get back to the original message. So you're always trying to break up that pattern, and that's what the Vision Air Sefer does. Towards the end of the... So we can leave the Vision Air Salad, do you think, fine.

Let's leave the Vision Air Salad. It'll keep coming back. It'll keep coming back. It does. We leave it for the moment. That is the sort of template for code, for encode of cryptography ever since. And as the historian, rather than the person who is an expert in codes, I would say that it's ever so much easier if you have the grid in front of you, which is on the website and which you can look at, or you can look at it in Simon Singh's book, it's not nearly as difficult to understand as it sounds when you try and spell it out. But it's beautifully, it has a beauty and elegance about it because it does involve a simple grid, but grid of letters.

You mean if people like me when look at it, they think they can understand it. And you can, and you can. Simplicity is the term you're looking forward. But more importantly, you can do it. I think it turns back to what Fred said earlier, that Fred could teach codebreaking to 13-year-olds. And this was something that geniuses of the 10th century wrestled with. Now, again, you can teach the Viginaire cipher to a 10-year-old, but it took somebody with the brilliance of Albertian visionaire to work this act.

Concepts which were incredibly difficult 500 years ago, to us kind of seemed trivial. But there were huge leaps of intuition to make these breakthroughs in codes and code breaking. Another one, Simon, was at the end of the First World War, there's a new method of code writing, as I understand it, called the Worn Time. pad. And as I understand it, it is still used, still used today as the code system for the London to Washington hotline. Now, what's the basic idea of the one-time pad, which is still being used between 10 Downing Street and the White House? Okay, the one-time pad is absolutely unbreakable. I just guarantee rock-solid, if I encrypted a message with the one-time pad,

and I sent it to Fred, and MI5 tapped our phone line, they would never, ever, ever be able to break it. And that's why it's so important and so wonderful. And what you do is you take every single letter of your message. You write out your message, so you've got this sequence of 100 letters, and you're going to shift every letter. Now, we're always talking about these shifts, but the shift for each letter, inside the alphabet. Inside the alphabet, is going to be completely random. Completely random. So before I encrypt the message, I might, let's imagine I've got a 26-sided dice.

I roll the dice a hundred times, so I get a sequence of 100 random shifts. And the first letter is going to be shifted by maybe 10 places, a second by 4, the third one by 18 and so on. And because every shift is completely random, the output is completely random. And the code breaker has absolutely no patterns. Almost the definition of randomness is something without any pattern or structure.

And that's why you cannot break it. But the shortcomings are, Fred. Well, let's suppose you're in Australia and I'm here. I have done this. I send you the cryptogram. What can you do with it? absolutely nothing until you know those random numbers I'd added so you can take them off. So how did they get, because we've still got the problem of the messenger?

That's right. So we've still got the problem of getting the, what we call the key, this random sequence to you. Now, in certain scenarios, that is not too difficult. And the London Washington Hotline is probably one of them. They can send them in advance, and then I just tell you which one I'm using. So you will have, say, a thousand random sequences that I've sent you earlier by some very secure means. You actually do put people on airplanes with armed guards and stuff? Well, presumably, yes. Well, that's what we would tell.

Yes, that's right. So the problem is that, if you like, the extra material you send to protect your messages is as much material as the messages themselves. And it's called one time pad because it's used once and then destroyed. And that's because of this beauty that both are, mathematicians take absolutely for granted, which is as soon as you repeat anything in a cipher, you run the risk of being decoded.

Repetition is the bugbear of any pattern. Any pattern. They're so good at patterns that if you use a one-time pad twice, someone will notice. And this undermined some of the Soviet codes. When you've got to generate randomness, I talked about rolling a dice with 26 sides. The way the Russians did it was they randomly typed on a keyboard. But it's actually very hard to randomly type on a keyboard because you typically do one letter with your right hand

or one with your left, one with your right, one with your left. And so that's not random. It might look random to a naive viewer, but in fact... It does look random to just naive, isn't it? Yeah. But, you know, the odd ones may contain an E, but the even ones probably won't because the E's on the left hand side of the keyboard. It's not random.

And because it's hard work to generate these random ciphers, the temptation is, oh, we'll make fewer of them and we'll reuse them. And once you reuse them, that's when you get caught. And people like the atom spies were captured because the Soviet codes were decoded because they were lazy and used non-random one-time pads and then reuse them. It's very difficult for humans to generate random things. Why not?

Well, I'll give you an example. You should take the banking ATM network where you have a PIN number, and the bank lets you choose your PIN. Well, if you go up to the ATM not knowing what you're going to do, and they say now put in a randomly chosen form, digit pin if you really do it random without any thoughts you're put in, you're forgotten what you put in

and you've locked yourself out the system. So you will have your own favourite what you might call random numbers and humans are very difficult, it's very difficult for you to generate a random sequence. You can catch children at school, math teachers do this all the time. They say this is your homework, go home,

toss a coin 100 times and write down the sequence of heads and tails that you've got. And the children said, well, I'm not going to bother doing that and they just write down a random sequence, heads, tails, tails, heads, heads, heads, tails, and they try and make it random to fall the teacher. But in fact, what they fail to do is that in randomness, you'll get long strings of heads and long strings of tails, and the children won't do that, and then they'll get caught out. I have to assume a lot of people listening to this programme know a great deal about Enigma.

Can we just bounce through it and come to the present day? It's my fault. We've spent a bit too much time in the middle of ages in the Renaissance. Can Simon just say, why Enigma was such an extraordinary thing, both the making of the Code de Chrebiose and the cracking of it, we're cheering at Bletchley, If you can do it briefly, we'll comment on to what's happening now. Right. I think the key technological development is radio. People are sending radio messages all over the battlefields, all over Europe, into the Atlantic. That's great for communication, but it's great for people who are eavesdropping,

because it's easy, you know, people are listening to Radio Waves now and listening to us. So it's very easy to pick up radio messages. So if everything can be intercepted, everything has to be encoded, and that's what Enigma's great for. It's sort of a mechanized typewriter for encoding. And it sort of takes an image. and it might have encrypted an E as a W the first time. The next time it comes from E, it might encrypt it as a Z.

So you can't tell what really represents E because sometimes it's W, sometimes it's Z, and so on. And it's got this huge key space as well. The enigma has lots and lots of different settings. So the British not only had to capture an enigma, they had to capture the setting for that particular message on that particular day. So following on for what Simon said about, we are not good at randomness,

it electrically and mechanically generates multiple patterns of randomness. So it lays randomness upon randomness, but in ways that can be undone, literally reversed at the other end. So somebody has a machine at the other end and that this machine has the capacity of reversing immediately all that complexity that you put in randomly when you encoded the message. The phrase is pseudo-randomness. Anything do I add for it before I move on to Mr Cox at Cheltenham?

No, that's fine. All right. Mr. Cox at Cheltenham in the 1970s, unknown until recently because of the information. What did Mr. Cox at Cheltenham do that was so important? Well, basically, he's actually done two or three things quite important, but I think the thing you're getting at is that if you look at all the cryptography we looked at so far, it's done between two people who have to share a secret. Like the one-time pad is difficult to operate because we need to share a secret.

And the introduction of something that's called publicly cryptography, which was the idea of Cox and Ellis at CSG and then later Diffion Helman publicly, is that there is no need, or you can communicate secretly, without sharing a secret. That it is possible for you to be able to decryp something, for me to be able to encrypt it to send it to you, but that we don't need to share a secret. So I don't need... Let's just spend the time that's necessary to get that right,

because this is very important. So in other words, we don't need a key, in that sense. We don't need a key to... And we don't need a message. Encrypt. we don't need to create to encrypt the information because it's a public, everybody has it. Think of a mortis lock.

If a mortis lock, you need a key to lock, I need a key to unlock, it's the same key. If you take a Yale lock, anybody can lock the door, you just shut it, you slam it. You only need a key to unlock it. Yeah. Right? So in the first case,

the mortis lock's rather like symmetric cryptography because we need to share the same key or copies, but the Yale lock is the equivalent to public key cryptography in the sense that anybody can lock the information up but only you can unlock it. And this as a concept was

what Cox Ellis and Diffy and Holman came up with. So just say once more why this was so fantastic. Simon, do you want to say? Yeah, I mean, if I want to send you a message, I scrambled it up according to a recipe, the only way you can unscramble it

is if you know the recipe as well. And that's for 2,000 years. everybody said the sender has a recipe the receiver's got to have the same recipe otherwise it doesn't work so that means if i want to buy something online from somewhere in america i want to buy the latest brittany spears album i type in my credit card details i hit send i scramble it up i have to run all the way to america give them the scrambling routine so that they can unscramble it and it doesn't really make e-commerce work so the wonderful thing about this idea that fred's described that cox invented and and ellis and so on is that i can scramble up a message

using a recipe, I've never communicated with the receiver ever before, and yet they can still unscramble it. And it sounds impossible. The analogy that cryptophers often use is, I want to send you a message, I put it in a box, I lock the box, and I send it to you. And you say, look, I can't open it, I don't have the key. So what I do is I put the message in a box,

I lock the box, I padlock it, and I send it to you. You still can't open it. What you do is you put your padlock on it and send it back to me. I've now got a box with two padlocks on. I take my padlock off. It's a box with one padlock left. Send it back to you. It's your padlock.

You can open it, open the box, and read the message. No key was ever transferred between us. The box was always locked, and at the end of the day, you could open the message. You could read the message. So this proves you do not need to share your recipe necessarily. And that's how e-commerce, you know, pay TV, mobile phones, all work on this technology. It's sort of fairly typical. This seems to have been

found here in England, but patented and money made of it by the two people who took it up in America. It's a really secret. That's a very annoying. The visionary cipher was broken by Babbage in the 19th century. Nobody ever knew about that for 100 years. Enigma was broken here in Britain. Nobody

ever knew about that for 30 years. Nobody knew about Cox's work for 20 years. It's a secret business. Well, never mind. Ever mind. Thank you. I've got to go now. Next week, yes, next week we're discussing the Battle of Somopoli, and we had a quiz, which I don't know, anyway, we had a quiz, I couldn't imagine it myself, actually. The winner, there were 40 winners, the one drawn out of their hat in full public view was William Rushby from Sainsbury and Wilshire. The dictionary is on its way. Thank you very much for listening.

We hope you've enjoyed this Radio 4 podcast. You can find hundreds of other programmes about history, science and philosophy at BBC.com.com.com. Radio 4.