The Rest Is History - 84. Exams

Episode Date: August 9, 2021

Even the memory of them is enough to make most people shudder. As thousands of British students wait nervously for their A-level and GCSE results, Tom Holland and Dominic Sandbrook are joined by educationalist Daisy Christodoulou to discuss the fascinating history of the examination. Your time starts now. *The Rest Is History Live Tour 2023*: Tom and Dominic are back on tour this autumn! See them live in London, New Zealand, and Australia! Buy your tickets here: restishistorypod.com Twitter:  @TheRestHistory @holland_tom @dcsandbrook Learn more about your ad choices. Visit podcastchoices.com/adchoices

Thank you for listening to The Rest Is History. For weekly bonus episodes, ad-free listening, early access to series, and membership of our much-loved chat community, go to therestishistory.com and join the club. That is therestishistory.com. Hello, welcome to The Rest is History. This episode is going out on the 9th of August and it's a time of great stress for families and particularly for students across much of the country here in England because tomorrow on the 10th of August both A-Level and GCSE results will be announced. it's not normally done like this but obviously circumstances because of the pandemic are very different and the same thing happened last year and it last year particularly served as a reminder of how

fundamental exams are both to education and how we measure education and so Dominic we thought it might be interesting to ask why that is and where did the idea of exams come from how did they evolve what's that everything has a history doesn't it and exams are no different um yeah I first I did my first exam Tom when I was almost eight so we had formal exams from the age of you know from whatever year that is at my school and and the headmaster would announce the results in reverse order at assembly i mean some eurovision some people some people may say that's that's bad i i thought it was brilliant actually i love because you always you always came top did you i uh well it's not for me to say it's not for me to say how i did but i used to walk i used to walk out of

that assembly with a massive smirk on my face, put it that way. Probably utterly, I mean, if there's anybody from my school listening to this, this will just confirm all their worst fears. Well, I think, Dominic, in that case, we need to unleash on you and your monstrous smugness. One of the country's leading educationalists, Daisy Christodoulou, author of three books um founder of a company that is kind of daisy basically your company just quickly explain it because it's it's kind of revising the idea of exams isn't it sure yeah so hi hi tom hi dominic um yeah so our company we're called no more marking um we have a different technique that we use for assessing essays, assessing extended pieces of writing. It's called comparative judgment. And it's not new. It's not something that we invented. It was invented back in the 1920s. But what we've been able to do

is kind of plug it into a piece of software, which makes using it much, much quicker. So what teachers do instead of reading an essay or piece of writing and marking against a rubric, what they do instead is they'll see two pieces of writing on screen, on the computer screen. They'll read them both and they'll say to themselves, which is the better essay? Which is the better piece of writing? And they make a series of decisions like that. And teachers from across the country will make a series of decisions like that. And then we combine all of those decisions to come up with a measurement scale for every piece of writing. So it's a different way of assessment. It's much more reliable, it's more valid, and it's pretty quick as well.

So exams is something you've thought a great deal about. I'm going to doubly embarrass you, first of all, by telling you what Amanda Spillman, head of Ofsted, how she described you. A fountain of energy and inspiration, Sherlock and Mycroft Holmes rolled into one. Wow. And Dominic, that couldn't be better

because we've got Sherlock Holmes coming up. Yeah, who's Moriarty in this analogy? Who's Dr. Watson? Well, I think I'm Dr. Watson. You're clearly, with your fiendish ability at exams and doubtless long division sons, you are Moriarty. The Napoleon of exams. The Napoleon, exactly.

But Daisy, also, I mean, essentially, we are your second favourite podcast, is that right? Because your favourite is our sister podcast. We Have Ways, done by my brother and Al Murray, all about the Second World War. Yeah, I'm a fan. I quite like it, yeah. You have a drinking game, is that right? You have a drinking game that you play

when you're listening to this World War-themed podcast? I think, you know, Tom's just throwing me under the bus here, aren't you, Tom? What do they do? They mention a panzer and you drink a pint of meths. I think their great phrase is, whenever they talk about an elite picked group of, you know, an elite group of picked men,

any sort of combination of those words, you have to drink. Tom, we should institute a rest-is Rest Is History drinking game or bingo card type arrangement, shouldn't we? But I think our episodes are so varied that it would be very difficult for people. But every time you mention something being sacral, that should definitely,

that should definitely be one of them. Shouldn't it? I don't think I repeat myself at all. I think I say something new every moment. Well, maybe we could throw this open to the listeners and see if they've got any suggestions for a Rest Is History drinking game. game but anyway we're being diverted um

daisy exams where do they begin so the sort of famous example of where they begin in the famous sort of prehistory is in is in china so they go back an incredibly long way in china uh much much you know earlier than the the we start using them in the west um so as early as kind of really about the seventh century they've got the origins of exams yeah um but the origins of exams in china uh being used as um a means of selection for the civil service so that's the the kind of really big history with them in China. And these are written exams, are they, Daisy? So they seem to start off as maybe an oral interview, but they quickly kind of evolve into something that is written and they quickly become quite systematic, quite standardised.

There's a whole kind of national infrastructure involved in administering them. So certainly you've got things in the West, perhaps in a bit more of an ad hoc way that might look like what we think of as exams, but nothing, as I say, as extensive or standardised as what's going on in China. And in fact, in the West in the 19th century, in the 18th and 19th century,

when a lot of reformers are looking at civil service, a lot of them keep going back to China and looking at what's happening there and praising it or critiquing it or thinking how we can do better. So John Stuart Mill, he talks about it in On Liberty. He talks about the Chinese exam system a bit there and the Chinese system of government. So it's something that when you get reformers in the 19th century thinking about how we should use exams in schools and for employment,

that they are going back and looking at what's going on in China. And is that why we call top civil servants mandarins? Yeah, it absolutely is. So there's a Jesuit, Matteo Ricci. He's one of the first, I think maybe the first Westerner to visit China to actually kind of see what the Chinese state is like. I think he goes there in 1601. He spends a lot of time preparing for it.

You know, he sort of learns Chinese and he spends a lot of time kind of finding out what there is to know know about china at that time and he is really impressed by the exam system the way they select uh the way they use the exams and select the kind of best and brightest to run the country um and he he writes about it a lot and when he comes back and that's again where a lot of people in the west are sort of catching on to it getting interested in it and yeah it seems to be he's the one who um therefore comes up with the term Mandarin for a civil servant and is really impressed with the quality of all these civil servants. So let's unpack this a bit. So 7th century China. I guess what you need to have exams, you need a bureaucracy, a sort of national bureaucracy that can kind of either roll them out or bring people in to take the exams.

But you also need a pretty high degree of literacy, don't you? I mean, people need to be able to, if there's a written element, you need an awful lot of people who can read and write. And in the West, certainly, that's not a lot of people. So I imagine even in China, you're talking about exams within quite a small segment of the population, aren't you? Because surely a lot of Chinese peasants can't read and write. Yes, it's really interesting. Yeah, really, really good point.

And so I suppose the 7th century is the very earliest origins of them. And they grow and grow, as I say. But yeah, it's definitely true. A lot of people who've written about it, including Matteo Ricci, have sort of reflected on the fact that if you look at maybe countries in the West, what they're doing at the time, China's instituting this big program of exams. It's pretty different. I don't want to come on this podcast and have a go at the heptarchy. I know Tom wouldn't be happy with me. That would be another tricky argument. But a lot of people who've written about this have reflected that as China's bringing in a kind of exam system at the same time as in England, you've got the heptarchy, which, as I say, on most podcasts, people will probably say it reflected badly on the heptarchy,

but I don't want to say that. You can't come on this podcast and be reading in English accents. That's very poor. So I think also the exams, these exams in China, they're in classical Chinese and they're on the Confucian classics, and classical Chinese is not the vernacular. So for everyone who's studying them, they're having to learn another language almost. I suppose it's a bit like, you could say it probably is a bit like Latin in medieval Europe. It's really interesting. It's a language that kind of isn't a vernacular, but serves

to maybe unite lots of different vernaculars. So as a kind of nation building, a sort of nation building system, it probably has that in common. And you're right, it's not something that the sort of Chinese peasants are doing, but the numbers do get up to being pretty big. I mean, certainly, you know, if we fast forward a bit, but by the 19th century, you are looking at two or three million kind of sitting the basic exams. Two or three million people. There's several stages of exams. There's a couple of different stages. There's like a basic local one.

Then there's an intermediate. And then there's the palace exams, which are the kind of top, top, top ones to become the kind of top mandarins. And in terms of people starting out at the basic, the basic ones, yeah, you know, it gets into the millions. Obviously by the mid 19th century,

the population of China is 350 million. Of those millions who are taking them, how many kind of, well, so how many get past exams and get places and how many reach the absolute top? So this is where it gets really interesting. So for anyone sort of struggling, thinking about maybe not getting their first choice exam at a university place now, or if they have got your first choice university place, the system over time got increasingly, increasingly harder to get these top spots. And in fact, lots of people have kind of implicated the failure of the exam system to reform and kind of provide more roles as one of the factors in particular the King dynasty in the 19th century, in, in, in some of its troubles and its downfall, because again, by the mid 19th century, you're looking at maybe two or 3 million people kind of setting out on that journey,

trying to do those basic exams, but the very, very top roles, uh, you're looking at perhaps, you know, um, it's, it's, it's, I think on average, it averages out about maybe like a couple of hundred. Okay. Um, and then the, the, you know, leave the top jobs aside for a minute, even the intermediate ones. So if you pass one of the intermediate exams, you can still get a kind of civil service role. But what's interesting is those numbers don't really increase over time and keep pace with China's population. So what you're looking at is, as I say i think the population mid-19th century is about 350 million and the number of kind of you know civil service jobs available there is about half a million but that hasn't increased in a couple of hundred years um even as the population's increasing

the people are going for increasing so you're getting more and more people uh taking these exams uh more and more people kind of studying, spending, and they're hard exams. You know, you've got to learn all these classical Chinese characters. You've really got to start from a very young age. You've got to put a lot of time and effort into it. And yet the numbers, you know, the numbers of jobs you're getting are fixed and are not really going up. So there's kind of increasing pressure on the system. And that's classic kind of revolutionary material, isn't it? Overeducated, disaffected middle-class kids. I mean, they're the revolutionaries down the ages.

Absolutely. So I think this is where it gets really interesting, really interesting parallels with today. So, you know, there's really interesting stuff being written at the moment by Peter Turchin, who has this quite statistical approach to history. And his argument is that you can pick out, and I don't know whether you guys are like this or not but you can pick out some kind of key metrics uh statistics over history and use those to predict revolution and his argument is that the most important metric i think he picks out like three or four but one of the most important i think he says even the most

important is um what he calls elite overproduction so the idea that you've got some systems in your society that are generating people with elite education elite aspirations but there aren't enough elite kind of roles or jobs for them to go into and he talks about modern america as an example that the number of senators has kind of stayed constant even as he's one of his favourite graphs is the number of law graduates that has really exploded. And I think China is definitely a kind of parallel here, China in the 19th century, that you've got this civil service, you know, sort of almost fixed number of roles, but you've got more and more people with that education maybe expecting that they

should get one of those and they are well educated they have studied for a long time they do have this knowledge of the Confucian classics and you know the jobs aren't there for them so so yeah really interesting to think of it in that light. So Daisy the people who study for these exams are they often the children themselves of civil servants? Because that's a kind of a feature of today, isn't it? That people who go to elite universities meet other people from elite universities and produce children who then go to elite universities. And so it perpetuates itself down the generations. So, again, some debate about this and again, lots of parallels with where we are. So there also seems to be the suggestion that over time, the kind of social mix of people get winning, you know, passing the exams know earlier on in the system the sort of 13th 14th century actually you do get a fair spread of of people from different different you know sort of different

different backgrounds whereas it really then starts to narrow and it does tend to be perhaps something it becomes a bit more of a you know hoops to be jumped through and people who kind of know the secrets and can can work out a way way to pass there's simply some some sort of suggestion uh around that that yeah the backgrounds people know they even have quotas regional quotas to to ensure that one region doesn't dominate uh yeah absolutely it's it's fascinating uh kind of how many parallels there are that it's such a long time ago and a different culture but i think so many so many similarities with i think what anyone working in exams in this country would really recognize there the most, the guy who has the worst exam result with the most consequences in history.

Yeah, we can't do this without mentioning Hong Zhukan. I think you've got to mention him if we've talked about the elite overproduction and exam failures and what happens there. So who's he? So he is, he's a kind of mid-19th century exam taker. He's one of the exam takers we talked about. And the other thing I failed to mention is, another thing that might make you feel grateful today is,

in China, it's not like you take these exams at 16 or 18, you fail and it's like, oh, just move on with your life. You can keep taking them again and again and again. And people do keep taking them again and again and again. And there's people who go on into their 40s, 50s, 60s, you know, they keep retaking them. And Hong Zhikan is one of these guys he's family of investment education in his kind of first round of local exams he comes first but then he gets

onto these later stages and he doesn't do as well and he fails them four times okay so he fails that kind of next step four times that step to kind of get you the the job and over this time he's been introduced to christianity there's a lot of american missionaries in china at that time so he's been introduced to christianity read some pamphlets and i think when he fails the the exam for the fourth time he has this vision which convinced him he's the younger brother of christ ah it's the typing rebellion yeah it's a typing rebellion yeah um and he inspires a number of people to follow him it's a typing rebellion which i think actually nowadays people say maybe it should be called a typing revolution or typing civil war. I think rebellion in many ways underplays it.

It's astonishing. It's, you know, it's I think between 1850, 1864, about between 30 and 50 million people die. And when it's most powerful, you know, he is running the typing heavenly kingdom, which rules over 30 million people. It's got a capital and it actually has the trappings of a state itself including it seems examinations um so uh examinations not yeah examinations not not of the Confucian classics but of a translation of the bible that he's made right this is a gift to you Tom Christianity finding its way even into exams absolutely well I was that's why i wanted to ask about it but i mean for anyone i mean anyone kind of you know failing their a levels that's that's quite a path to go isn't it yes slaughter millions killing 50 million people yes um do you think um so so this influence is

presumably the west you you said that Should we take a break now? And then when we come back, look at the evolution of the exam system in the West. Would that be a good plan? Sounds good to me. So if you'd like to stop writing, please, lay down your pens and we'll give you your second paper in about three minutes.

I'm Marina Hyde. And I'm Richard Osman. And together we host The Rest Is Entertainment it's your weekly fix of entertainment news reviews splash of showbiz gossip

and on our Q&A we pull back the curtain on entertainment and we tell you how it all works we have just launched our members club if you want ad free listening bonus episodes

and early access to live tickets head to therestisentertainment.com that's therestisentertainment.com. That's therestisentertainment.com. Welcome back to The Rest Is History. The exam starts in a couple of minutes. Pens at the ready, please. We are going to be talking about exams in the West. So Daisy, your first question. What about monks and things like that? So presumably they had tests of some i mean there must have been some sense of a test before the gladstonian reforms of the 19th century

so i think the big thing the sort of medieval religious kind of equivalent is um they have the disputatio disputation medieval disputation where you're having monks who have uh kind of these big kind of oral arguments with each other about big theological questions and I think that's probably the forerunner of today's viva and things like that as I say I think what you kind of lack in the west before the sort of 18th 19th century is anything that's really kind of standardized anything that's that's and not just standardized but also in this way the China thing comes in it's scalable so people even nowadays like the idea of a viva and i think back in the 19th century when the royal society of arts was coming up with um its its exams they

said well these written exams are good but to be really sure the results are right you would also want to call the students in for a viva you know if you want to talk to them and this is again one of the challenges and it's again one of the challenges china had it's one of the challenges that exam people in exams today have you know my day job is something we do all the time that what do you do when you want to scale up so you can definitely see how having a viva might be the best way of doing something so daisy just for listeners who are not british yeah not familiar with the term what's what is a viva so i think nowadays you have them for phds and you would you know it's where you're having basically a kind of an oral interview so you've got um maybe a panel of people who are experts in the subject you've maybe written your

dissertation you've written your essay and you're being questioned on it in depth and so if you want to do something that's as i say really really in-depth and thorough and it's very hard to cheat on, that's the thing people say about Vivas as well, quite hard to cheat, not impossible, but hard to cheat, then they kind of feel like a real gold standard. But they're very hard to scale up, obviously. And also, they are even, you know, they have their benefits. They're still kind of hard to standardize. And one of the things, you know, one of the tensions, as I say,

running all the way through the whole history of exams China, US, UK is this issue of standardization and reliability and consistency and it pulls you in different directions so in China they have this essay that's called the eight-legged essay and it's an essay that it's really set out really strictly kind of what you have to write in each bit. So it's got these eight parts. And over time, it just becomes quite pedantic and a bit of a hoop to jump through. And it's so similar when you read the critiques of it, the critiques people have today of exams that are very nitpicky and rubrics are very nitpicky. And, you know, actually, the history GCSE exam is often one that really comes in for this kind of criticism of if you don't use the word significance or bias, you just can't get

the mark. Whereas if you use the word significance, you get the mark, but people will say, you know, you could have written nonsense. Well, as I say, this is a debate that's going back a really long way. You know, to what extent, how standardized you have to be to get the reliability. And if you have, you know, too far down that route, do you sacrifice kind of the meaning or in the technical terms, do you sacrifice the validity of the exam? So how do exams start to be introduced into, is it Germany that blazes the trail?

So again, yeah, Germany, they introduce, actually it's Prussia. So it's Prussia. So in 1788, they introduced the abitur and that's a kind of exam that's required for university entrance and that's still used today and then again not only uh lots of people in britain looking at china but they're very much looking at prussia so prussia state reforms at this moment are having a really big impact in in in the uk and in fact um it's not just kind of the the school exams that are having a big impact but prussia i think after the you know skip forward a century after the um franco-prussian war

in 1870 even before then there's been a lot of british people looking at the prussian army looking at the british army thinking hang on a minute and so not only do prussia have the the school exam 1788 but they're also probably their professional army is something that encourages the British to get rid of the purchase of commissions in the army we were doing we we were doing our statue walk Tom when we came across the Duke of Cambridge who opposed all this yes no he said that so yeah there's a huge controversy in the mid-19th century so the the guy who's really more in favor of them of getting rid of them is Garnet Worsley.

And he's kind of one of the figures behind the Cardwell reforms of, again, sort of late 1860s, early 1870s. He very much thinks, you know, you need to professionalise army recruitment, the Purchase of Commissions, it's got to go. So I think the Purchase of Commissions does go in 1871. And Worsley is satirised in Gilbert and Sullivan's Model of a Modern Major General. And if you remember the lyrics of the Model of a Modern Major General, it's about this general who knows this fantastic amount of stuff, you know, the facts about the square, the hypotenuse, that kind of thing. And, you know, it's based on Walsley,

who was very keen on professionalising and having more formal recruitment and education for army officers and not just having people buying commissions. So, yeah, you've got it. It's a real theme throughout the 19th century, both with the army and in education. And Daisy, exams are all about this. I mean, you need a state, don't you? It's all about state power and imposing a kind of uniformity, isn't it? And sort of, you need somebody at the centre who decides what it is that people need to study and what they need to be assessed on. You need an apparatus to roll it out. You need everyone buying into the same system. So this is what's really interesting is that you could argue that actually in England, maybe you don't. And so the really interesting thing, not just about exams, but the whole English education system in the 19th century is it centralises very, very late. You get central kind of state oversight and state funding of education generally very late.

And that's because of religion. That's not because there isn't a will. There is a will from a lot of different groups. But the thing stopping it is that almost the state comes along too late and you've got a lot of different religious groups who have been self-funding their own schools, their own exams. And so, you know, there's this reluctance to say, well, if you're going to put taxpayer funding in it, then I'm a nonconformist. And that means that my tax money is going to go to the Catholic Church. I'm not happy about that. And even as late as the 1940s, you've critics of the butler act who are objecting and the phrase they use is it's rome on the rates okay and that's as late as the 1940s so that is actually one of the reasons i think you know sort of again why in england people often look now at the fact we've got all these different exam boards and often you know i'll talk to sort of people from outside the english system say this

is mad how can you have all these different exam boards? And I think even Michael Gove, when he was education minister, was a bit unhappy about it and felt it was leading to dumbing down, a race to the bottom. But the reason we have all these different exam boards is kind of a legacy of that. And John Strickland also talked about this on Liberty. He's another person who's really in favour, massively in favour of education, massively in favor of exams but because he's also in favor of liberty he's really wary about the state imposing exams imposing a curriculum imposing schools and he comes up with all these ways around it both in terms of the funding you know it's a book on liberty spends a lot of time talking about how you're going to get education and funding of education without tyranny um and so one of the legacies of this is you do have still kind of the remnants of all these local exam boards that sprung up in the 19th century and what you also had is um lots of universities so you're saying like does it need a state well the first you know the thing that a lot of people in england think of as the first big exams were actually set in 1858 by

oxford and cambridge and they were set they were called local exams and they were set for students who weren't part of the university for 16 and 18 year olds that they could have a certificate to show what they'd studied so actually in England you know you don't have the kind of state the state sort of running it in the way you might think but so before 1858 how how are people getting to Oxford and Cambridge so so you you go to a you know, a Harrow or Eton or whatever, or rugby. Is it just, you know, you... So from what I've seen, it's just a bit, again, it's a bit, it's very English, a bit ad hoc, a bit hodgepodge. You've got, you know, colleges that have certain links with certain schools and they will have certain places for them.

So you don't have to do an exam or anything. There are exams. So I think Trinity is the first Cambridge college to bring in exams exams i think they do bring them in in the 18th century so so yes you do have some examples of exams coming in earlier than that but you have still a lot of as i say sort of these interviews and these there's famous examples you don't know how true they are of you know the the the the lord of whoever's son turning up and the question on his interview is well how's your father and he gets in you know so this is what i'm saying about standardization you don't feel like the interviews are being run on a sort of standardized protocol with a rubric marks out of

whatever um so i think the exams are coming in gradually and some colleges are bringing them in before others um but it's it's it's kind of patchy and ad hoc and and then and then not state not not centralized you start to get kind of london universities which are reacting against the oxbridge monopoly with its emphasis on classics and starting to kind of introduce science and absolutely and so they're pulling things like that idiocy idiocy yeah so the judgment around what you should study is also huge and so even when people say okay we'll have exams that doesn't end the debate because the debate is then well what should we have on the exams and you have

the famous uh northcote trevelyan report um which is i think 1854 so again really similar to all the time all these things are happening and northcote trevelyan that's about entry to the civil service and that's all about how you know you've got there's too that's about entry to the civil service and that's all about how you know you've got there's too much nepotism in in civil service entry and how you need to put it on at this more professional recruitment basis and you need to have exams but that just opens up the kind of words well what should be on the exam and there's been a lot of modern critiques who said well in the end they you know they went too far down that route of classics and kind of not enough down the route of the sort of professional knowledge and training that you might need to be a good civil servant.

You can't come on this podcast and talk about going too far down the route of classics to Tom Holland. But again, that mirrors what you see in China. Because in China, you know know the exams are the Chinese classical Chinese and the Confucian classics but that went well for several centuries it did it perhaps you know by the time it ends in 1904 1905 doesn't end particularly well yeah all right no point the bit when they collapse into rival warlords that didn't work so well it's a detail, a detail. And people are saying then in the 19th century,

you have a lot of Chinese reformers who are saying, we have to reform these examples, reform the content of what's on them. We need more science, we need more technology. And actually, weirdly, that ends up being the same thing that people are saying about the British Civil Service. And I think a lot of people would still make that critique now. They would say there's not enough of that. And there's a very interesting story as well about the, I think a lot of people would still make that critique now. They would say there's not enough of that. And there's a very interesting story as well about the, I think,

Haleybury, Haleybury College, which actually its full name, I think, is Haleybury and Imperial Service College, which is, I think, set up to recruit civil servants for the East India Company. Which Clement Attlee goes to, right? Clement Attlee. Absolutely. And there's a talk that actually it should also form the basis,

what they did, they should form the basis of what you do to recruit for the British Civil Service. but i think that gets kind of that doesn't quite work out and you end up with something that's much more much more sort of classical classical literature classical languages you know something that's much more focused on that um so yeah the content you're absolutely right to say that the content of what's on exams is something that really is thought about and And Daisy, you mentioned 1858. Am I right in thinking you have something there, you have an exam paper? So I have in front of me.

You see, listeners, how desperate Dominic is to be examined, to show off his knowledge. It's appalling. It's like he's eight years old in his school. Let's scrap the chat and get straight to the questions. Come on. So I have a facsimile. So, Daisy, I hope you heard that Dominic is volunteering for this. I'm going to send him a message. is yeah let's scrap the chat and get straight to the questions come on so i have so daisy i hope

you heard that dominic is volunteering for this i'm i'm gonna say i think we should i think it should be competitive i think it should be competitive the loser bows out of the rest is history so i have a copy in front of me of the 1858 examination papers so these are the by the university of cambridge so these as i say are kind of the these these local exams they're set for for for students to sit in in their school in these local or these local centers they're set by the university of cambridge and marked by them um and it's sort of seen yeah as a sort of a bit of a landmark moment these are a really really really big moment in the history of exams in in england and uh they've all the subjects i'm just flicking through them.

We've got English composition. We've got English history, which again, Dominic, I'm sure you're going to be keen on. Yeah, I'd be confident. I'm very confident on English history. We've got geography. We've got Virgil. Geography.

We've got a paper on Virgil. I'm not doing Virgil. I wouldn't say that. We've got absolutely tons here. I will point out there's one interesting thing. I remember when I started teaching, I taught English. I used to be a teacher. And you notice that the exam papers would always really reflect what was in the news and I remember starting to teach in 2007 and all the all the students I think they'd written a kind of coursework essay on the smoking ban because the

smoking ban had just come in and so they were all giving their pros and cons of the smoking ban and what's really interesting is that that seems to just always been the case that you do that. So I've got here Wednesday, December 15th, 1858, English composition. So no smoking ban questions. But what's the first question? Give an account of the late Indian mutiny. Oh, I love that. I'd be well up for that. Don't do that one. I mean, that might take a bit of time, Dominic. Go on, Dominic. Give us a look at the Indian mutiny. I refer you to William Dalrymple's appearance on The Rest Is History just a few weeks ago.

So you've got number two, contrast the life of a soldier with that of a sailor, both in peace and war. OK, that's good. That's a good one. That's quite broad. Number three, write a letter to a friend in Australia announcing your intention to emigrate and asking for information. I think that's so 1858. I just think that's so 1858. And finally, what do you think the fourth one is? Discuss the change produced in the habits of the people

by railways. Oh, wow. That's a very typical. So the Australian one, I would do that. You know what I'd do? I'd write to Mr. McCorber. He would be my friend. He emigrated to Australia, didn't he? I'd write to him. Yeah?ber he would be my friend he's he emigrated to australia didn't

he i'd write to him yeah i could tell him there's something you want to pick magwitch i would say something had turned up um i would i would ask him for repayment of the loan that i had made just before he left the country i've got it all planned and i'm sure if dickens was marking i'd do very well no f derivative so i would say the english composition probably hasn't changed that much in terms of it's a topical topic you've got to write you've got to find find something happening in the news right about it like i remember in 2012 everything seemed to be about the olympics you had to write something about the london olympics i looked up some from the 1950s and there's some then that are all about the festival of britain you know tell us what you

think about the festival of britain so that is actually one that maybe hasn't changed as much but some of the others i think have really changed um i think geography is very much um oxbow lakes do they have oxbow lakes i did no no there's no oxbow lakes i'm just looking at the geography paper and it's all drawing oh no so they've got explain the following geographical terms, but Hogsbow Lake is not one of them. But there's a lot of, you know,

where's, draw a map, where's this, where's that. You see, that's proper geography. That is geography. To me,

that's not outdated at all. That's completely reasonable. No, but geography's gone wrong now because it's all about, you know, dreary stuff that nobody, capital cities and flags.

Draw an outline map showing the coastline of Europe from the mouth of the Danube to the mouth of the Rhine. Mark the chief rivers and the ranges of mountains between those two rivers and the coast. I think, you know, you have to submit your papers here. Describe accurately the situation of the following places. Genoa, Londonderry, Mecca, Rio de Janeiro, Singapore. Mecca. I mean, that's a controversial one. Describe in words the course

of one of the following rivers, mentioning the chief towns upon its banks. The Thames, the Severn, the Rhone, the Danube. This is why Victorian Britain was great. But these sort of questions... But these sort of questions endured for quite a long time though, didn't they? Because I've got here,

I downloaded overnight an exam paper from 1950, a history paper, right? So this is a Scottish leaving certificate exam. So kind of similar age group, I guess, Daisy. I was really wanting to ask Tom some of these questions. So question four, for example,

what changes did Henry VIII make in regard to the church and for what reasons? That's interesting. It's a Scottish question. That's such a massive question. Isn't it? Next question.

I mean, how about this? Trace the chief steps in the development of South Africa from the Great Trek to the Union Act of 1909. How do you think you'd do with that, Tom? I might look at another question. What were the reasons for the collapse of Napoleon's empire? Yeah, I could do that. Could you do that?

Could you really do that? Overconfidence. Describe the domestic problems. And the glories of the British Navy. Describe the domestic problems that faced the British government from 1905 to 1914 and explain the measures they took to solve them you do that i could probably have a crack at that because i've listened to you on the causes of the first world war that's a strange death of liberal england isn't it suffragettes trade unionism ireland ireland yep and then here's one for ted valence

who we had on talking about magna carta outline the events in john's reign that led up to magna carta and explain briefly the importance of that document making sure to give the dates of king john's reign yeah well he didn't know so he would have done badly despite being a professor of history but what's interesting about that it's a scottish exam paper but so many english questions um so scottish listeners will say this is an example of the totalitarian you know yes so there's been a move away from that there has there has hasn't it and it's all kings and battles i mean this is 1950 so it's 1066 and all that i mean they parody that don't they well they do that's 1066 and all that the victorians would recognize that as completely

reasonable but probably exams have changed far more in the intervening period in from then till now than they did in the hundred years previous do you think that's true daisy uh i would say yeah i think that probably is true yeah i think there's some truth to that so again i sort of spent a bit of time in the cambria assessment archive looking at the o levels in the 1950s um and again yeah the history if you compare them to the 1858 ones i think there are a lot of similarities um certainly the geography and the history are much more kind of lists of facts looking at that. So Daisy, I guess the argument, say, for example, of a Daily Mail columnist. Where could such a person be found?

Mentioning the name. But there is a kind of harumph every time GCSE A-level results come out that they're not as difficult as they used to be. Are you in a position? I know what Tom wants you to say. Everyone knows what Tom wants you to say, Daisy. Have exams got more difficult? Have they got easier?

Are they basically the same? Can we measure them? This is a great question and one that I could probably spend an entire podcast on. It is really surprisingly hard to answer the question. And in fact, we've looked at exams really for a very kind of historical, political lens and educational lens. Another interesting way of looking at exams actually is as a branch of measurement. And there are actually some really interesting overlaps between, you like the development of the history of temperature um the measurement of temperature the measurement of time that there's actually some really interesting overlaps there but one of the the challenges with

with with the measurement and trying to work out if standards have have have changed i'll say um is there's just a number of issues so so one is what we want to do is look at the questions, don't we, as we've just done and go, well, that question's really hard. There's a professor of history at the moment who couldn't answer that question. Ergo, exams are getting easier. But the big problem is you can't just look at the questions. I've got this whole book of questions. But how do you know that the kids who took them then actually, how do you know they didn't do really terribly how do you know actually they just left blank papers they didn't have a clue so we're all sitting here it's kind of rote learning isn't it i mean it's kind of rote learning

dates of kings and queens the next critique is is that the next critique so so first of all the first reason why it's hard is you can't just look at exam paper you need responses and we don't have enough responses that you have these archives of question papers but you don't have enough archives of responses. So that's one problem. The second problem is what you've just said, is that exams have really changed their focus. And so someone might look and say, well, okay, so a kid didn't do well on these questions about the dates of whatever. Someone might look at that and go, well, I don't care. I think history is more than that. So you might have a very talented history student today who wouldn't do well on that exam, but they're still better at history because our notion of what history is has changed. Now, I'm not sure how much I would buy that with

history, but I would probably buy it with other subjects. What we now call biology, which in here is probably more zoology, that's changed its focus. And I think entirely legitimately, I don't think anyone would argue with that because we just know more now. So the other problem is you've got subjects changing. That makes it harder to compare. And the other problem you've got is that these exams from the past are often aimed at a very different slice of pupils. So these exams of the past were often being taken by a fraction, a segment of the population. The modern GCSE is taken by everybody. Even the old O-level was only really been taken by 20, 25 segment of the population. The modern GCSE is taken by everybody. Even the old O-level has only really been taken by 20%, 25% of the cohort.

And that changes the way you set up your grading. Yeah, because the process is so much broader. Right. So just imagine if you've got, you know, when you've got grades, you've maybe got five or 10 or nine or however many grades you've got to award. If you're going to award that grade set to the top 10 of the population it's going to look very different the top grade is going to be awarded to a very different fragment of people than if you're awarding a grade set to 100 of the population that's just going to happen yeah so you've got to factor all those things in so

i'm getting all my excuses in up front but notwithstanding that you can do some interesting comparisons even with all of those caveats. And the person who carried out a really, really interesting study into this is actually a colleague of mine. And he used this technique I talked about, comparative judgment. So his name is Dr. Chris Whedon. And he did this paper back in 2016 with some colleagues. So it's not just him, a group of them. And what they were trying to address is the question. So to address one of these to, so to, to, to address one of these problems about exams changing their focus and content,

they picked a subject that hasn't changed as much. They picked a mathematics A level where you do have a bit more stability in, in the kind of questions being asked. And they also did have some archive responses and they use this technique, comparative judgment to compare the difficulty of the question and the responses over time. And they actually got maths phd students to judge it so it was students who kind of didn't know about the english system the students didn't know why they were taking part in this they didn't know that it was about standards over time so the really clever study

they set up and i will say now up front that it won the 2016 british educational research journal paper of the year so it's a really you know I'm not just saying because it's my colleague it's a really clever paper really well done and they were comparing maths a-level responses from 1964 1968 1996 and 2012 and they were saying have the standards has the stat have the standards changed and what they found was that between 1964 and 1968, there wasn't much change. Between 1996 and 2012, there wasn't much change. Between 1968 and 1996, they found the standard

that would have received a grade E in 1968 would have got a grade B in 1996. Wow. That really surprises me. That's the punchline. Goodness. So I was expecting, because you were almost setting it up there, and I thought you were going to say,

oh, this debunks the idea that things have changed. This shows that actually we're clever enough. But actually, that's a massive drop of standards, right? Right. Again, I do want to caveat it, is that over that same time, you had many more people sitting the A-level. And so, yes, I think it is fair to say grades have changed in meaning i think that is absolutely fair to say and i'd almost say you would expect that if you have that many more people kind of taking them but actually that doesn't necessarily mean that underlying attainment is is falling or rising it does mean the grade has changed its

meaning right it doesn't mean we're getting stupid or cleverer it's not enough there to say that but it does mean the grade has changed in meaning but going back to specifically history um there's no doubt that what people study has changed you know very dramatically so looking at this paper that the 1950 paper and thinking about what i did at gcse because i was one of the earliest gcse. I mean, I can remember we got to do an empathy exercise. I mean, empathy was very big in our teaching. And, you know, I had to write a letter pretending that I was a housewife in Hamburg, explaining why I was going to support the Nazi party in 1933. And, you know, that's the kind of exercise that would have been

unthinkable in 1950, let alone 18, you know, obviously in the 1860s or something. I mean, so you're not really, I mean, maybe you are testing like with like with maths. You are comparing like with like, but you're definitely not doing that, are you, with history? Because history, the emphasis within history has changed so much. So that's absolutely why it's a challenge to make these comparisons. And that's why you can do it with something like maths.

As I say, I think you could do it with English composition, as they call it then. And I think there have been a few attempts to kind of do that as well with writing standards, because the subject, the question you're asking isn't changing as much. But I think you're absolutely right with something like history or geography. It would be more difficult. You would. Yeah, it would be. It would be more difficult because of all the reasons that you said. And again, not just history there'll be a number of other subjects where that the content of what you you're assessing has changed so much so so daisy um we're probably running out of time

and uh the invigilator will take away our papers but um just two questions prompted by kind of recent events. One is, do you think the experience of the pandemic has sharpened people's sense of the value of exams or led them to question it? Do you think that the system will be the same as it was in 2019 when we come out of the pandemic? And the other one is just to go back to the question of elite overproduction but maybe just maybe the first one first sure so i think what you see here is i find there's quite an interesting division between people i would say who are kind of on the on the outside of education and people who are actually probably teachers and in the system which is that there have been quite a few calls i mean a group of education ministers former education ministers came together recently and said the pandemic's shown us that exams don't work and we need to abolish gcses now most of the teachers i talked to think the pandemic's shown you that exams for all their

flaws are really necessary because the we've had two years without exams and i i can't imagine how you would look at that and say what we've had in those two years has been an improvement um it's been really messy. And I think the big issue you have is there's lots of legitimate critiques of exams. Absolutely, there are. And some of those, as I say, date back to China. The great problem you have when trying to replace them with anything, it's a bit like democracy. There's a lot of critiques of democracy at the moment. What are you going to do instead of them? And when you, all of the alternatives you come up with seem to have all the flaws of exams plus some new ones. So what you've seen this year and last year with the teacher assessment is the teacher assessment.

Many issues with, I think, but one of the big issues you have with it is that poor students, disadvantaged students, students with behavioural issues, students with special educational needs all tend to do worse on teacher assessment than on exams and people are often really surprised when i say that they say no it's the other way around you're wrong it must be the other way around and it's not there's a a huge research body on this um and it's to do with similarly big research bodies on human bias and the thing i say teachers are biased not because they're bad people they're biased because they're human and and lots of things that involve human judgment do have those those biases built into them so as I say that's one of the the the the best evidence findings in in the assessment literature so that the the great problem you have if you want to rely on just the kind of pure teacher

assessment we've had in these past two years as a we've had it because of necessity is that it doesn't really give you that level playing field and exams certainly have their problems and i'm not pretending that exams are the perfectly level playing field either but again it's i'd say of all the different alternatives they're the ones that are probably the best so i think there are lots of people who are saying well the pandemic shows that we've got to change things but my argument would be the pandemic shows us that when we can't run exams, we have real problems. So I would say I think we need them and I think we'll still be using them in years to come. And so just on the back of that, kind of looking at the broad context of what we've been talking about generally.

Do you think in the future that exams will serve the cause of social mobility or entrench existing divisions? So that is a great question. And it is hard to say. And as I say, I don't want to pretend that exams are perfect and that they are this perfect engine of social mobility. Again, I think it is really interesting to look back to China, where I think you can argue that there's a phase in their history where they probably were working as you would want them to work. And then there was a phase where they weren't. And I think you can see something similar probably in the modern Western system of exams that you can see there are ways where they do work as they're meant to there are ways where they don't um i i think there are real issues around the content of what should be on exams and whether what is on exams is preparing students for

the world i think that's absolutely legitimate question so i think that you want reform of that that seems to me it can be sensible i think the other um big issue you have as well is um yeah i think there is an issue around if they do just become if you like box ticking hoop jumping exercises and i think if you do get too much almost kind of over reliance on rubrics i think that can decay confidence in them. And I think that also means they're probably not doing their job of actually giving students the knowledge and skills they need to get by in the world. So they become almost something that's separate from the education.

They can become a bit detached from it. They become a means to an end on their own. I think that is a real problem. And I think that's the kind of thing, we talked about this happening in China, where you then can get that dominance of them. And I think there's that feeling that, well, you know, maybe private schools have this secret source where they can just get their pupils to kind of jump through these hoops. So I think it is really important that they're fair, and there's a level playing field. And I think it's really

important they're perceived to have that. And I certainly think there's ways we could reform them so that they do a better job there. And I do share the concerns of people who say if we don't make some of those reforms, then it's not just a problem for the exam system, but it is a problem for society because of that outsized role that exams play in our life. But Tom, if elite overproduction is your concern and you think that elite overproduction leads to... Well, no, it's not a concern. ...leads to unrest, right, then greater social mobility in exams wouldn't solve that. It would just mean a different part of the overproduced elite was satisfied and...

Right, so Peter Turchin, his argument is you don't want to be making... You don't want to be getting all these meritocrats striving for the top role. You actually need some way of damping down competition. Yeah, that would be his argument. He takes that quite kind of counterintuitive argument. I would say the implication of that argument is actually just stop educating so many people.

Because that just, I mean, you could laugh. Robust is no nonsense. But that's surely the obvious conclusion, right? Unless you're going to magically create the jobs for the people to go to then the if if elite overproduction is your fear then the issue is elite overproduction i.e stop producing so many elite people or produce more elite jobs well you can't just unless you've got you are you blessed with the magic wand that would create elite jobs out of nowhere well that's why i need to become dictator.

The other side of it is, I suppose the other side of it is if you're not worrying about elite overproduction, but you're worrying about you want to get the very best talented people into jobs and into the top jobs. And if you want to do that, that would be good for everyone. Because if you have the most talented people in the top jobs, then they're hopefully going to run things more competently and make better decisions so that would be the alternative kind of sort of meritocrat argument that you do want the competition you do want people competing you want exams that do find who the most talented are and do also uh you know give them the education they need and and that will be good for society but also i mean there's a difference isn't it but i mean just to go back to whether exams are an end in themselves the more more people are educated, the more they're in a position to create jobs.

And so the economy grows. That is absolutely gibberish. Not at all. Not at all. Because if everything is funneled through the prism of examinations and, you know, there are certain kind of, I don't know, Oxford Ecologies or something, that's seen as the absolute end. And of course, there's a finite number. But if people's education is better across the entire spectrum of society,

then people have the knowledge. Then the magic wand does appear. Tom Holland's magic wand. It's not magic. It's sensible policies for a happier Britain. I am right, you're wrong. The best educated cabinet we ever had was the Wilson government of the 1960s.

I mean, they were all Oxford dons. They were all super educated. Now, nobody, I mean, not even their greatest fans points to that government and says... No, but you're looking at it again through the prism of elites, through people going to Oxbridge. That's not the point.

The point is that if you look at it, there are exams you have to do. There are certain exams you do spectacularly well, you get the best prizes. That's the Chinese system. I mean, that's off. If you look at things entirely through the prism of getting to universities, that's how we would look at it. But there is another way of saying that you take away the whole apparatus of exams. And it's about learning. It's about knowledge. And it's about getting skills and understanding and perspectives that then enable you to kind of create your own your own context i think that's a really interesting point i'm going to throw something else into the mix here it was another sort of

china and john stewart mill point which is when they were talking about exams for the civil service in the 19th century john stewart mill said um i wouldn't want the best and brightest in the civil service he said if the exams meant that all of the best and brightest went to work in the civil service that would be terrible. And he said it would be terrible because you would then have nobody outside the civil service will kind of have the capacity

to solve their problems because you just have everyone in the civil service kind of doing it. He also then said that you wouldn't have anyone with the capacity to check they were doing the right thing. So how would you check

if there was any corruption or anything? And he also said, and this is his China and he also talks about Russia, the problem you would have is you would have like a self-perpetuating oligarchy and in some senses when exam systems get too good and again you could argue this for china they they they do become self-perpetuating and it gets really really hard to uh bring in any reform and change the content on what you're being examined that's the argument behind michael young's book about the rise of the meritocracy. That's the argument behind Michael Young's book

about the rise of the meritocracy. So he's the person who, Toby Young's father, wrote the 1945 Labour Manifesto. He effectively coined the phrase meritocracy. But he was talking about that as a warning. He was saying you create this elite of meritocrats who are very, very well educated, who then think, you know,

they're different from aristocracies or oligarchies, because they think, I earned this, you know, I'm morally better than other people, because I worked so hard, and I'm so clever. And, and that argument is around a lot in politics at the moment that we've created, that there is this sort of self conscious, middle class, highly educated, meritocratic elite, you know, the winners and losers of society you know who think that they are not just luckier and cleverer than other people but they are morally more deserving because they earned it through their own efforts through passing exams through precisely this point you know what this entire episode shows is that we've just degenerated into a rambling

what it shows is that there is almost no topic more calculated to generate argument and disagreement and intense feelings on the subject of exams. And Dominic, I'm going to... Yeah, you're going to have the last word. You're going to have the last word. No, I'm not. I'm going to take your exam page. But Daisy, I can't thank you enough

because I really think that the historical perspective does shed light on the timelessness of these, you know, people's kind of very ambivalent attitudes towards exams. As exemplified by the way that this entire conversation is disintegrated. So I can't thank you enough. Yeah, thank you, Daisy. That was fascinating. Thanks, Tom. Really, really interesting.

Thanks so much. And we will be back on Thursday with, as I said, beginning of the programme, Sherlock Holmes. Bye-bye. Goodbye. Thanks for listening to The Rest Is History. For bonus episodes, early access, ad-free listening, and access to our chat community,

please sign up at restishistorypod.com. That's restishistorypod.com.