Freakonomics Radio - 14. Do More Expensive Wines Taste Better?

Episode Date: December 15, 2010

They should! It's a cardinal rule: more expensive items are supposed to be qualitatively better than their cheaper versions. But is that true for wine?

So, Levitt, a college friend of yours once told me that your favorite meal during college was a dill pickle, beef jerky, and grape soda. Is that true? I did indeed have that for breakfast. But to tell you the truth, it sounded better before I ate it than after. Steve Levitt, my Freakonomics friend and co-author, an esteemed economist at the University of Chicago, has an extremely refined palate. All right, so you got beef jerky, you got dill pickle. What are your favorite foods? Like, what are your favorite places? Like, if you could drive across America and pick any place to stop and eat, what's it going to be? You know, I love the Billy Goat Tavern. It's the cheap place that was made famous in the 19, on Saturday Night Live with the cheeseburger, cheeseburger, no Pepsi Coke. Anyway, they have an incredible ribeye steak sandwich.

Pretty much the cheaper the food, the better. There's almost no fast food that I don't adore. So KFC? Yeah, I like KFC, burgers, Chipotle. I'd kill for Chipotle. So how would you describe your palate? Probably underdeveloped. You know, but it's good.

I mean, the thing is, it's a wonderful, wonderful gift to like cheap food. I mean, some people just happen to like expensive food. And then they're unhappy most of the time, or else they spend all their money on food. I mean, some people just happen to like expensive food, and then they're unhappy most of the time, or else they spend all their money on food. But if you just, by chance, are born loving cheap food, then you can eat everything that you love. Now, how much do you like wine? Wine I do not like at all. From American Public Media and WNYC, this is Freakonomics Radio. Today, why wine experts should just put a cork in it. And why having an untrained palate can save you lots of dough.

Here's your host, Stephen Dubner. Good wine, we're told, is the province of smart, superior people. They taste things on a whole nother level than people like you and me. It's like a little minerality. Freshly cut apple. George Clinton. This funky sort of. Baby vomit. But I have a question for you. Is that superiority deserved? The wines that experts love, the ones that bring out the natural beauty of the grape but aren't too funky, they cost a lot more. But are expensive wines really that much better than cheap wines? Or is it possible that developing your palate just mucks things up, complicates things? Maybe we'd all be better off if we had taste buds like Steve Levitt.

Kill for Chipotle. After graduate school, Levitt was invited to join an elite club at Harvard called the Society of Fellows. A junior fellow like him was paid a modest salary to work on his own research with few obligations other than a formal Monday night dinner with the senior fellows, who were some of the most remarkable scholars alive. People like Amartya Sen and Nobel Prize winning physicists and whatnot. And you sit around the table, and I believe the table was Oliver Wendell Holmes' table initially, and he gifted it to the Society of Fellows. Over dinner, they engaged in witty, learned conversation. They ate venison, other fine food, and they drank expensive wine,

bottle after bottle. To this budding young economist with the beef jerky taste and the grape soda budget, all that pricey wine, it wasn't doing him any good. He had a thought. Innocently, I mean, I was young. I didn't understand how the world worked, I thought, like an economist. I suggested that perhaps we should have two tracks at the Society Fellows. There would be the drinking track and the abstinence track. And for those of us who chose the abstinence track, because the cost of the wine was perhaps $60 per meal, that over the course of 50 weeks of the year, that would work out to be about $3,000 and they could add $3,000 to the paycheck of those on the abstinence path. And did you have any other people in your abstinence camp, or was this just Levitt?

Well, you know, it didn't really get that far because the reaction was quite negative. Levitt's the kind of person who likes to use data, not a personal agenda, to make his arguments. So he set out to get some data, wine data. The wine they were drinking at these dinners cost five or ten times what a cheap bottle of wine cost. Was it really five or ten times better? He hatched a plan. The Society of Fellows held wine tastings from time to time.

He suggested that the next one be his to organize. So I worked with the wine steward to select two excellent bottles of wine, expensive bottles of wine, probably close to $100 bottles of wine. And then I went to the liquor store that was down the street, and I said,

can I have the cheapest bottle of wine you have that was the same grape? I don't remember which grape it was. Levitt used four decanters. Into the first decanter, he poured one of the expensive bottles of wine. The other expensive wine went into decanter number two. In the third

decanter, he poured the cheap wine, which cost around $8. In the fourth decanter, he repeated one of the expensive wines. So as far as the people knew, there were four different wines, and these were all wines that were coming out of the wine cellar of the Society of Fels. So you're tricking them from the outset. You're leading them to believe that the fourth, the cheap wine, is also from the wine cellar. I don't, you know, yes. They swirled them a bit, sniffed them, and sipped.

They wrote down their ratings. As he looked at the numbers, Levitt's cold, economist heart warmed. The data could not have cooperated more completely with my hypothesis. So for starters, the four wines received almost identical ratings on average. Although they were widespread among individuals, on average, it's tallied it up, people did not prefer the expensive wines to the cheap wine. On top of that, and this was the thing that I was hoping for and dreaming of, but didn't believe it actually came true, it turned out when you made among individuals, if you compared how differently they rated any two of the wines that they had, it turned out that by a small margin,

people actually rated the same wine from the same bottle but presented in a different decanter as being the most different among the two wines. So the two wines that were absolutely identical, when you looked at the gap between the ratings the two wines that were absolutely identical, when you looked at the gap between the ratings that an individual gave to those wines, the gap was bigger than they did between the other wines, which actually were different. A few minutes after the tasting was over, Steve Levitt shared the results with the senior fellows. The jovial mood in the room

suddenly went dark. People realized they had been tricked, that there had been this cheap wine, the same wine was in twice, and they really realized that the nature of the game had been somewhat different than what they thought. And when they heard the results, that collectively they had no ability to identify wines, they were not happy. And in particular, there was one senior fellow, so one of the professors at Harvard, who was quite outspoken about his knowledge of wine. And he loudly announced that he had a cold. Otherwise, he clearly could have made the distinctions. And he stormed from the room and left the party prematurely. What was his discipline?

I think for the sake of anonymity, I should not reveal that particular piece of information. He was a humanist. He was not an economist, in other words. No, no. The opposite of an economist. So what does Levitt's evil little experiment teach us about wine? Maybe not all that much. It wasn't a very scientific tasting, really. Perhaps the Society of Fellows was just having an off night. The humanist had a cold, right? Or maybe this was just a group of people who didn't know as much about wine as they thought they knew. You'd never be able to pull this kind of stunt on wine experts, would you? On my team, we have a master sommelier, two master of wine candidates, four people that have been in the trade for many years.

These are sophisticated wine professionals. That's Brian DeMarco talking about the people that he gathered for a blind tasting of his own. Before we get to that, let's hear a little bit more about Brian. helping customers, consumers, private collectors, and retailers and restaurants decide what wines to put on their list, what wines to collect, what wines to sell. And I have a small import business and wholesale company, and we distribute wine that we find all over Europe and South America in New York City. Cool. So you're almost a wine agent then, yeah, in a way? More than just an importer? In many ways, yes.

Brian DeMarco is one of the people who decide what we drink. He goes to France, Italy, California, tastes wine fresh out of the tank. So he determines what's good or bad without any critics whispering sweet ratings into his ear. Then he puts his money where his taste buds are. He writes a check. Now, according to DeMarco, he makes as much money selling a $15 bottle of wine to a restaurant or shop as he does selling a $50 bottle. That's true. DeMarco is an honest broker. His job is simply to find wine that you or I would want to drink. Because if you think about it, there are thousands upon thousands of bottles to choose from. Just picture the rows of bottles lining the shelves at every wine shop.

You kind of, sort of, maybe think you want to buy a Merlot. So do you pick the one with the pretty flowers on the label? Do you go with the one that Robert Parker, the high priest of wine ratings, awarded a lot of his Parker points to? Or, like a lot of people, do you let price be your guide? If you believe even a little bit in the free market, you'd have to think that expensive wines cost more because they taste better, right? Brian DeMarco wanted to know how much people were tasting the dollars when they drank an expensive wine. So, like Steve Levitt, he conducted a little experiment

with some of the people who worked for him. As DeMarco said, these were no amateurs. We did a tasting, a brown bag, and we had the exact same wine in both bags. And we told them that one bottle was a $50 bottle and to write their reviews, and we said another one was a $10 bottle to write their reviews. And, of another one was a $10 bottle to write the reviews.

And, of course, they're both $20 bottles according to retail, what they would sell at any retail store. And then we reversed it. And we said now the $10 bottle is really the $50. And everyone liked the $50 bottle better in both circumstances because they perceived that the price they had, either they thought there's something they're missing. When in real life, they're like, these wines are so similar. Yeah, but they're different. And so a lot of people call, two or three people said, is this the same wine? And we said, well, that's for you to determine.

And the more they thought about it, the more they intellectualized it, the more they decided there was differences to the wine. So this wasn't a wildly scientific experiment either. But to Brian DeMarco, the message couldn't have been clearer. When people know a wine is more expensive, or even think it is, it tastes better. Now, obviously, this idea doesn't apply just to wine. A house that costs $500,000 ought to be five times better on some level than a $100,000 house, the size, the construction, the schools, the neighborhood. So is a wine that costs $50 five times better than a $10 bottle?

Or is it even better at all? Coming up, we go looking for hardcore empirical evidence that expensive wines actually taste better than cheap wines. And we look beyond price to prestige, to wine ratings, the awards, the whole shebang. Because if your wine or your whole wine list wins an award from a magazine like Wine Spectator, it's gotta be good, right?

Right? Swirl, sip, and spit. We're back with Freakonomics Radio from American Public Media and WNYC. Here's your host, Stephen Dubner. So tell me your name and kind of what you do or how you describe yourself. My name is Robin Goldstein. I write about wine and food. Basically, my book, The Wine Trials, has been my, I guess, principal outlet for writing about wine. But I've also been publishing academic papers on topics of taste from a cognitive perspective and an economic perspective,

usually co-authored with colleagues from different academic fields. I've been exploring the neuroscience side of it a bit. I've been exploring kind of the behavioral side of it. And in particular, I'm interested in price signals and how people's knowledge of price affects their experience of wine on the most basic sensory level. Now, first let me just ask, I've heard good things about this lovely little restaurant in Milan called Osteria L'Intrepido. You ever been there? I've actually, I've been to the restaurant, but it's actually located in my friend Giuliano's former apartment in Milan.

The restaurant really, I wouldn't say it's great. Mostly they serve leftover pizza, and their wine cellar consists mostly of some leftover bottles of Montepulciano d'Abruzzo from three weeks ago. Meaning it's not really a restaurant, is it? It's an apartment. So it's just an address, really. Osteria L'Intrepido doesn't exist. It's a fake restaurant that Robin Goldstein made up. Why? Well, it's a strange story. Goldstein's research and writing on wine made him skeptical about critics and awards. He believed that so-called experts were at best subjective and that they carried way too much influence. He wondered about the awards that magazines like Wine Spectator gave to restaurants for their wine lists.

Did an award like that really mean that the wines at that restaurant were excellent? So he invented Osteria Intrepido, or Fearless Restaurant. He created a fake menu, a fake website, and a fake voicemail message saying the restaurant was closed for vacation. As for the Osteria L'Intrepido wine list, Goldstein made that up too. He included several expensive wines that Wine Spectator itself had given bad reviews in the past. One of them was a 1982

Brunello di Montalcino, which the magazine had given 67 points for a D-plus rating, calling it barnyardy and decayed. He listed another vintage that Wine Spectator had reviewed as unacceptable, sweet and cloying, and smells like bug spray. Then off his application went with the fake wine list and a real money order. My hypothesis was that the $250 fee was really the functional part of the application. In other words, that the entire awards program was really just an advertising scheme

and that it was being fraudulently misrepresented as an exercise of expert judgment by Wine Spectator. I see. Now, was a little piece of you expecting that when you applied for this, that they would send someone around to drink some of your wine or eat some of your food? Well, of course, that's the experiment, right? So I didn't know. I mean, I wasn't sure going into this that I would win an award. There were two questions being tested here. One was, do you have to have a good wine list to win a Wine Spectator Award of Excellence? And the second was, do you have to exist to win a Wine Spectator Award of Excellence? And so I thought that it was quite possible that my experiment would fail.

He didn't fail. I am calling from the Wine Spectator magazine in New York, calling to congratulate you and let you know that your restaurant was selected as a Wine Spectator Restaurant Award winner. Hopefully you did receive some information from us, but I'm calling to follow up and see if you might have an interest in publicizing your award with an ad in the upcoming issue,

which comes out in August, as we do have a lot of international business travelers who may be coming to Milan. And what was the name of the award that you won for your fictional restaurant? The Wine Spectator Award of Excellence. So congratulations.

It's awesome that you're a winner. Thank you. I, too, could be a winner, presumably. Yeah, if your wines were bad enough. Have a great day. Bye-bye. In August of 2008, a new organization called

the American Association of Wine Economists held its annual conference in Portland, Oregon. That's where Goldstein revealed his fake award of excellence. The press drank it up. From the New York Post, wine mag humbled by hoax. According to the LA Times, Wine Spectator was now drinking a hearty glass of blush. Wine Spectator vigorously defended its award system. The executive editor said the magazine never claimed to visit every restaurant

and that it did its due diligence on Osteria L'Intrepido, looking over its website and calling the restaurant, but that it kept reaching an answering machine. Robin Goldstein, for his part, was convinced. The wine system was fundamentally flawed. If a fake restaurant with a wine list that included bad, expensive wines could win an award of excellence from one of the most prestigious wine magazines in the world, who are we supposed to trust?

My takeaway is that expert sources in the media are trusted too much and that they're prone to abusing their positions of power as a way of making money. So the phenomenon where what's really an ad is posing as real expert judgment is very problematic for consumers because consumers really put trust in these magazines. We put trust in experts. There's so many fields out there that we don't know as much about as the experts do. And so we use experts as an information intermediary,

as a proxy for good judgment in an area that we don't know as much as the experts are supposed to. And when we trust experts too much and they sell their awards to entities that are really their customers, that's quite problematic. Robin Goldstein also had an academic paper to present at that conference of wine economists. The paper was called, Do More Expensive Wines Taste Better? If the Ossorial Intrepido stunt was just a stunt, and if the Steve Leavitt and Brian DeMarco blind tastings we heard about earlier were just unscientific tricks, well, Goldstein's paper was the opposite of that.

It gathered up data from 17 blind tastings that Goldstein himself organized. The data included more than 6,000 observations from more than 500 people, from amateur wine drinkers to sommeliers and winemakers. He tested red wines, whites, rosés. The prices ranged from $1.65 a bottle to $150 a bottle. It was as rigorous as you could get. And what did Goldstein learn? That overall, people liked expensive wines less than cheap wines. When you don't know what a bottle of wine costs, apparently you don't know how good it's supposed to taste. Even the most

expert tasters could barely tell the difference between expensive wines and cheap ones. It's unsettling, isn't it? Buying a bottle of wine shouldn't be as complicated as buying a house. But thanks to the layers of experts between us and the grapes, we've got performance anxiety. Wine isn't supposed to be a drag. It's a celebration, a beloved recipe, civilization in a bottle. You are drinking the handcrafted fruit of some farmer's vines that may go back hundreds of years, grown under the same sun that's been shining for billions of years. Wouldn't it be nice to drop the pretense, to set aside the ratings and price and just drink? Ryan DeMarco, the wine importer?

That's what he's really after. Who are we to tell the Connecticut housewife that that oaky Chardonnay she's been slamming down for the last 10 years from California, she's not deriving pleasure from that. She surely is. But there are so many other things. It's like limiting your musical notes to only playing F and C notes the rest of your life. It's only eating cheeseburgers and hamburgers the rest of your life. If you want to try other foods and other cultures and other things, if opening this bottle of Rioja somehow takes you to Spain and makes you feel like you want tapas, then it's done its job. And whether it's a $15 bottle or a $30 bottle is irrelevant at that point.

I mean, then you're getting into nuances of good and great. But I think the bottle of wine is the ability to either change your day from an alcoholic standpoint, you know, getting to a certain point where you're like, okay, this is, this is. Or for people like myself in the trade, it's something you have with food. I mean, growing up in an Italian household, we weren't drinking great wine. But to me, wine wasn't alcohol. Wine was Carlo Rossi in a jug that was poured in the juice glasses from my grandfather. And that was what you had with dinner. Drinking to me was Budweiser and Jack Daniels, you know, sneaking out on Friday night to go drink. Let me just ask you, we're sitting

here in a radio studio in downtown New York with kind of mucus colored sound padding and artificial light. And there's basically nothing good about this space. There's no air, there's nothing. Soulless. But transport me for a minute. So tell me, Brian, the room that you would like to be sitting in right now and the food that you'd like to be eating right now, and most of all, what the wine is with that food

and what that wine tastes like to you. I think a crisp Sancerre would be perfect right now with some sort of croque monsieur and maybe, I don't know, a salad with a little vinaigrette would be kind of perfect. Not too decadent, but definitely has its place and maybe have a backup bottle on ice. Has your table got room for another guy there? You're sitting with me right now. has its place and maybe have a backup bottle on ice.

Your table got room for another guy there? You're sitting with me right now. This pressed wood table, I think it can hold an ice bucket. I'm pretty confident. Freakonomics Radio is produced by WNYC, American Public Media, and Dubner Productions. Our producers include Andrew Gartrell, whose taste guided this podcast, and Susie Lechtenberg, who views life as a blind tasting. This podcast is mixed by David Herman, who never touches the stuff. Our executive producer is Colin Campbell. Subscribe to this podcast on iTunes, and you'll get the next episode in your sleep. Thank you.