Magic: The Gathering Drive to Work Podcast - #544: GDS3 Check-In

Episode Date: June 8, 2018

The Great Designers Search 3 was plugging along behind the scenes when I recorded this, so I checked in to give you some info on how it was going.

I'm pulling up my driveway. We all know what that means. It's time to drive to work. Okay, so today I'm going to talk about The Great Designer Search 3. So one of the things I wanted to do is along the way as we did it, I wanted to sort of do some podcasts so you can sort of see it evolve. Now given there's a time delay, now I'm not quite sure. I'm going to talk more about the process, and I'm going to talk about what happens. As we get into, things are delayed, and I don't want to necessarily make a podcast that gives things away. But I do want to talk a little bit behind the scenes to give you guys a sense of what is going on. Okay, so, originally, we were going to do the, I think I did a podcast for this a little bit. We were going to do the, I think I did a podcast where I talked about this a little bit. We were going to do the Great Designer Search a couple years ago.

And we had come up with a, one of the things that's always been contentious about the Great Designer Search is the multiple choice test. That it is very non-subjective. And so one of the things was, the reason we have a multiple choice test is we need to narrow down from many, many people this year, over 3,000, down to something that we can grade, which is about 100. And we need something objective that, like, a computer can grade because we don't have the ability to individually look at everything. You know, I can't let 3,000 people take design tests and grade them. It took me a whole week to do 100 tests. Obviously, 3,000 would have taken me a year plus to do and then I couldn't have done any magic

but anyway so two years ago we talked about trying to do something a little different it required us building something to do it it turned out we weren't able to build it and anyway eventually we said okay let's just run it the way we've run it in the past. Let's, let's do a, uh, you know, we had talked

a lot more about maybe making a more of a video based thing. And in the end we're like, okay, um, there's a lot, there's a lot, this is not an easy thing to do. The running joke I had on the great designer searches, um, people always say, when's the next one? And I'm like, when I forget how much work the last one was. So what's basically happening was we decided to sort of follow the way we had done the first two and not reinvent the wheel. We did the multiple choice test. We didn't do video. You know, we just kind of did it the way we've done it.

But we didn't want to have a few changes. I wanted to walk through sort of the changes we did make. And then explain a little bit behind the scenes how this works. Okay. So the way that it works is that for the first great designer search, I was a judge. Devin Lowe was a judge. Aaron Forsythe was a judge. And Gleemax was a judge. Gleemax was. we wanted someone just kind of,

if you guys know American Idol, the Simon Cowell, someone who's just blunt and to the point. And so we ended up making a character called Gleemax, which is, if you know your magic history, it's the alien brain in a jar that runs R&D. I wrote Gleemax, and the whole idea of Gleemax was to be humorous, but really blunt and to the point. Like very short, snippy, and to the point. And then for the second grade designer search, I did it with Ken Nagel, and then we had two rotating guest judges is how we did it.

This time, we decided it would be nice to get representation of vision design and set design and play design. So what we did is I represent vision design, Eric represents set design, and Melissa Datora represents play design. And then, because I like having guest judges, we had one rotating guest judge rather than two rotating guest judges. The reason I like the rotating judges is I like having other people involved and hearing from other people and just getting to see different aspects and stuff. But it's a lot to do. It's a lot. I mean, for those that might not know,

basically what happens is when we get a design test in, the judges have to grade every single card and grade the people. So that can be a lot. Like the design test, for example, was 10 cards each for eight people. So that's 80 cards plus eight people. So that's 88 things you had to write. And the thing on the people is a little longer. So I mean, it's a lot of writing. It's a lot to do. So I'll get through how we do that in a second. I'll walk through sort of our processes or process, or I guess processes is one of the ones. But anyway,

we decided that we wanted to have a little more representation of the different aspects of R&D. And then, oh, and the way you'll see, hopefully you've seen this by now, it's,

we color code the judges so you can tell who's saying what. You know, I'm always in green, Eric's always in blue, Melissa's always in red, our guest judge is in orange.

Oh, we have one other person that I added, which is a new time to this thing, is Eli. Eli Schifrin is the rules manager. And so I added him in, and the idea is he doesn't talk all the time. He just pops in when there's either a rule issue or a templating issue. So he's kind of there for some technical reasons to talk about stuff, but he's not required to talk about every card. He only talks about cards where there's something relevant to talk about. But as understanding the rules and templating are important for designers, I thought it was good to just have somebody saying, hey, neat idea. Yeah, I don't

know if we can do this. Or, well, we can do it, but wow, this would be a lot of work. Is this one card worth that amount of work? And a lot of times, anyway, it's just some practical stuff to get a sense of more hands-on, how would you actually do this if you were doing it at Wizards? Okay, so what happened was, a little history behind the show is, both the first and second great designer search were done live, which means when we put up the results for everybody to

see, you know, the contestants were reading the judge comments at the same moment that the audience was. And the way it worked real quickly is each time we've evolved a little bit. The first great designer search was every week for five weeks in a row. And then we realized that was a little too fast and didn't give us enough time to do the turnaround. So we ended up making it two weeks. So every other week they did the challenge so that it gave us, they had sort of a long weekend to do it. And we had enough time to grade it and get it up and post it.

And then for Grade Designer Search 3, we kept the basic premise of how two worked from a scheduling, but it was more like one in that there were more individual challenges, where two, you had to build a world, and it was all connective. The challenges flowed together, and these are more individual skill testing challenges. Okay, so the way it works is... Oh, sorry, I'll get that in a second. So what happened was we were planning to go live. We were planning, but then it turned out, for just some behind-the-scenes technical reasons,

there are some additional... Like, one of the things about doing the Great Designer Search is because we are actually getting somebody there are just some rules. Each one has had a different set of rules we had to follow depending on different criteria at the time. And based on, I'm not going to bore you with the boring details, but there are some things that made it so we, if we wanted to do it live, we'd have to cut down the amount of material we produced.

I mean, we would have had to produce it anyway for the contestants. And it just felt wrong to me not to show off all, like, my belief is it's fun to read along, to see all the comments, see all the judge comments. And I didn't want to sort of produce it, have all the material, and then not be able for everybody to see all the material. So we had a decision to make, which was if we wanted to go live, we had to cut down the amount of stuff the audience would see. Or if we wanted the audience to see everything, we had to build in a delay. And so I was thinking, my wife and I, for example, watch a number of reality shows.

For some reason, I thought about Amazing Race. For Amazing Race, you don't know, it's a reality show where teams of two race around the world. And the point I realized is

it's not as if when you watch the show they're currently racing around the world. It's over. They've raced around the world. They don't tell you what happened, but it's already done.

There's a time delay. And there's a few shows like Big Brother that are literally live, so it's hard to not do those live. But a lot of reality shows sort of do the show and then show it later. And so we decided, okay, let's build in a gap. We'll run the show. We'll have all the material. But when it's time, you guys will all see it in order. You'll see the show, just to be able to lay it a little bit.

It won't be live to you, but it being live or not being live is kind of secondary. It doesn't really matter. For all intents and purposes, when you see it is when it is live for you. But what that means is we're running things behind the scenes, and then when we've banked enough things and we're caught up enough, we will start showing it. The one thing that it might let us do once again i'm recording this before this has happened is it might let us run it actually as a weekly thing rather than as a bi-weekly thing originally it's going to be bi-weekly because it's live and we have to run

it bi-weekly to be able to do it but if we bank up enough and start it a little bit later we might be able to run it consecutively so it's it's every week you're seeing what's going on, even though for us it actually took two weeks. So the way it works is on Thursday. So the very first Great Designer Search, I think we gave them the weekend. I think we gave them Saturday and Sunday. And then for the Great Designer Search 2, we decided to give them Friday, Saturday, Sunday. This time we decided to give them a little more. So we were giving them Thursday.

They get the test usually like noonish on Thursday, the next design. So what happens is, sorry, I'm jumping ahead of myself. Okay, so, what happens from the beginning is, on Thursday, we,

we started with the design test, which we judged, and then on Thursday, we send them all the judging for all the contestants. So, not just for themselves,

but for everybody, because part of being good at the designer search is seeing notes that other people are getting and being able to follow what those notes are. And then we also give the next challenge. So for example, the first thing they'll see is here's the notes for the design test and here's challenge number, design challenge number one. Here's the notes for the design test, and here's design challenge number one. The reason we moved it back a little bit is, A, we want them to have time to read everything, because we think it's important to see all the comments on all the different people.

So one of the reasons we gave them the extra day was to give them time to read everything, and still give them the same amount of time to actually do the design. It's funny, I got some requests from the players saying, oh, could you give us more time so we can read it? And I'm like, you are getting more time. This is the most time any great designer searches had to build their sets. And in the early part of it, I've actually been having to make a little less number of cards just on our end to be able to grade them and on their end to be able to make them. As we advance, I'm going to be making them making a little more cards. The last few challenges will go up in the number of cards because as we go down in the number of people, we can do that.

Oh, by the way, there's one other judge that is not a judge you all will see, but is a judge behind the scenes, which is Mark Gottlieb. Mark, you guys might know him as the one-time rules manager. Mark is now a manager that manages much of the set designers. And so he manages the designers. And this position, he would be the manager for this position. So he's also involved.

Okay, so what happens is Thursday, we send out a letter. Noon-ish. We send out a letter saying, okay, here's all the judging. Here's the next design challenge. And then all the people make sure they reply back that they got the letter. And then if anybody has a question, they will write me questions. Usually I get questions on Thursday, sometimes early Friday, and usually once I answer that. Sometimes, by the way,

I'll occasionally send out FAQs, frequently asked questions, where if enough people are asking something, sometimes I'll just write it up and I'll send it out to everybody saying, okay, I'm getting this question. Let me just answer for everybody. Here you go.

I did FAQs for the design test and I did FAQs before we started the first design challenge. I haven't done one since then. And the way the design challenge works is there's just parameters that we build in for what they have to do. Usually there's seven or eight parameters. I talk about sort of how many cards they have to make, what rarities they have to be, what colors they have to be, what car types they have to be. many cards they have to make, what rarities they have to be, what colors they have to be,

what card types they have to be. I usually, usually they design for our standard legal set and I explain the context that, you know, usually the cards are all in the same set. So they have to feel like they come from the same place. There's different parameters from challenge to challenge, depending on the challenge. But I spell it out and then they have until Sunday night. So they get it Thursday afternoon, they have until Sunday night. So basically 11.59pm Pacific Standard Time, they need to turn in their test. Oh, one of the funny stories is, so J Treat is one of the top eight. On the first design challenge, we come in and everybody's turned in their test but Jay. We're like, what's going on? So we write to

Jay and we go, Jay, you turned in your test. He goes, oh no, no, no, I turned it in. And we're like, okay, we're not seeing it. He goes, oh, let me just send it again. And he sends it again. We still don't see it. And so we're like, what's going on? And so he sent us a test message. And we get the test message and he, and he's written to us before from this account. So like, we know we got stuff from his account, like what is going on? Why?

Um, and eventually what we realized was, um, on one of his cards, one of his card names was, um, I think fetish, but, but in a, in terms of like of like a shamanistic spell. But anyway, it didn't get through the filter at Wizards. And so we realized that, oh, it was being caught in the filter. And the reason we weren't seeing it was that the Wizards filter was pulling it out. So anyway, he changed the name of his card and all was fine. Anyway, I thought it was funny. Okay, so on Sunday night, they send us in their

design test. Then what happens is in the morning, so Stephanie Mitchell is my assistant for the Great Designer Search. She's doing a lot of the there's a lot of logistic work to do and she's handling the lion's share of it.

And so what happens is she collates all of the design tests and then she makes two versions of them. One is a collated version that's alphabetical and one is a blind version, which means it's not in that, it's not in alphabetical order and you don't know who the people are.

Some judges like to take it blind, some don't. The reason, by the way, that some like, some don't is some of the judges, and I'm in this camp, we give notes to the people and a lot of the things that I look for as a judge is are they listening to the notes we gave them? One of the things that's really important, one of the skills that I consider vital to a good designer is, okay, you do something. Hey, maybe you don't know something.

Maybe you did something wrong. That's fine. But then you get a note and do you listen to the note? Do you listen to what we say and adapt? And so the reason I like knowing who the people are is, I'm not just judging them in a vacuum. I'm judging them against what the notes were they got. Other people do it a little bit differently.

And so we have available both a blind and non-blind version. I think most of the judges are actually great at blind. I think Eli and I are the ones that tend to look at the names. The other thing that's very funny is because I'm a word guy, I've also figured out the naming conventions each of them use. So the other reason that I decided the names was okay for me was I kind of was starting to pick up on how they named their cards. So I was, because I'm a word person,

I was able to tell who designed what based on their names. So I was, because I'm a word person, I was, I was able to tell who designed what based on their names. So anyway. Okay. So Monday morning, Stephanie sends that out. So we have one week for everybody. So what they have to do is they have to write a note on every single card and a note on each of the contestants as a whole as sort of a sum up of what they think. each of the contestants as a whole is sort of a sum up of what they think. And then we, and then if you need an introduction, the normal judges do the introduction in the first episode. The guest judges do the introduction in the episode that they're the guest judge.

So if you're a guest judge, in addition to writing the critiques of the cards and people, you have to write an intro. So then they have until the following Monday at 9 a.m. to get it into me. And then I spend Monday and Tuesday compiling all the stuff together. Because when you guys see it, it reads as if all of us are talking with each other. I do a little bit of editing. I'm the last one, so a lot of it's in mine where I make sure that, like, if somebody says something

someone else says, we kind of reference, like, oh, yeah, yeah, as Eric said, or I agree with Eric or whatever. So that it sounds like we're all in the same room

even though we're not. So anyway, we have a week to turn that in. But on Thursday of that week, we get together in the afternoon. And what we do is everybody

takes the latest challenge and then orders the people from best to worst in their opinion. Then we compare notes. So each challenge, two different things have to happen. One is we pick the top three of the challenge, who's the number one, two and three of that challenge. And

we have to eliminate somebody. Now the elimination is done as all your work, not just that. Who wins the challenge is of that challenge. And we have to eliminate somebody. Now the elimination is done as all your work, not just that. Who wins the challenge is just that challenge. Who gets knocked out is not just that challenge. It's up to date all the work they've done. So for example, if you do really amazing on the first week

and really bad on the second week, you might not get kicked out just because you did bad on the second week because you did really well on the first week. So we're trying to find the best designers, obviously, and so we're looking at all the work combined. Obviously, the way the design challenges work, we're testing different skills. It's quite possible that we hit a particular skill you're bad at, but as long as you've demonstrated in other areas that you have some strength, then there's less chance you get eliminated. Obviously,

the worse you do, the greater the chance you get eliminated just because if you do really bad on a challenge, that means at least one challenge you've done bad on. So anyway, the judges discuss. We always invite all the judges, including Mark and the guest judge. The guest judge is more involved in picking the winner than the loser only because all they've judged is the latest stuff, and the rest of the judges, we've judged all of it.

Some of the time, some of the time we're really on sync. We just did one recently, which was Design Challenge 2, where we were all over the board. In fact, it's comical to read. And you guys, I don't know if you've read it. I don't know if it's been posted yet at a time you heard this. But it is comical in how, like, there's just, I like this. I hate this. This is good. I don't know if it's been posted yet at a time you heard this. But it is comical in how, like, there is just, I like this. I hate this.

This is good. I don't like this. This is flavorful. This is not flavorful. So it was a top-down challenge. Number two was a top-down challenge. So there's a lot of, I guess, subjectivity to top-down design. So anyway, we will then pick the top three, and we will pick the person who gets eliminated.

It is possible to both be in the top three and be eliminated by the way because you could have I don't expect it to happen until later in the process but I could imagine let's say we get to the final week there's four people and one person does badly but wow they've been rocking it the whole time

they might still make the top three even though they are the fourth. And that means one of the top three would get booted. So it's possible to be in the top three and get booted. It's not super likely, especially early on. It's possible, that's all I'm saying. So then they pick, so we then pick somebody. And then Monday and Tuesday, I compile everything. Also, I'm in charge of the challenges.

Normally what I do is I write up a challenge and I show it to the other judges and just make sure they're happy with the challenge. Eric had made the design test. I've been making all the challenges. With input from the the design test. I've been making all the challenges. With input from the other judges, but I've been making them. And then it's a matter of figuring out what exactly we're asking for. Because different challenges have different nuances. I've definitely been...

In some of the past GDSs, we've asked for more cards. I've sort of capped it for the first... I think the first three challenges, we're only asking for eight cards. Because that's a decent number of cards, but not so much that we can't grade them. And then the fourth and fifth challenge, I think, are going to be more cards

just because there's less people. Then, so what happens is, I have Monday and Tuesday to write up all my stuff, or to edit everybody. And then I do a little bit of overwriting and stuff. Once again, I want to make it sound like we're all in the same room. And then if somebody, I also, I'm back reading people, so if somebody makes a mistake or misreads something,

you know, like I remember I had a card where Eric said tapping. I'm like, I think he means untapping or whatever. I go to Eric, I go, you say tap, but I think you mean untap. And he goes, oh, yeah, yeah, I meant untap. I can change them or like Eli says instant, but he meant sourced or something like that. I'll catch those and fix that up. Anyway, then Tuesday night, I hand off my file to Chris. So Chris Gleason is my editor.

He's one of the editors, or he's the main, does all the editing on the website. I normally, he's the person I hand off to with my article. So I've been working with Chris for quite a while. So we give it to Chris and Chris has a tight turnaround because he needs to get it back to us by noon on Thursday so we can get it off to

the candidates. And he does. Usually what happens is I write the whole show, if you will, and then Chris edits the part that needs to go to the contestants, the designers. Because when I do the show, it's written as if it's for all you guys.

When we send it to them, it's just a compilation of the judge comments and the design tests. There's elements that would get added when you guys will see it, where I, hey, welcome back, stuff like that. That's taken out for the contestants. So then I get the stuff from Chris.

Then either Wednesday night or Thursday morning, depending on various factors, I will email the person who got cut, who got eliminated, and let them know individually. Like one of the things I don't want to know is I want the person who gets cut to know ahead of time and let them, because what happens is we send the judge comments to all the people that took the design test because we want everybody to see all the comments, including the person who got eliminated. We want them to see the comments. They just don't do the next challenge.

And anyway, we just, just out of pure politeness, it's a lot of work and a lot of, you know, I want to have great respect for the people who are doing this, the great designer search. Also, by the way, we, as a thank you to them, we made shirts and we made play mats and we sent them some goodies as sort of a thank you for participating. The thing to be aware of is one of the things we try to do in the judging is we want to be educational and constructive, but we're also trying to be pretty honest. And so it is, I know it is hard. The great designer search is hard. Just to be aware, the amount of time we give them is not a lot of time.

The challenges we give them are pretty challenging. And, you know, part of being an R&D is getting a lot of feedback and a lot of criticism. And one of the reasons we're doing this is we want to see how people sort of stand up to criticism. But at the same time, we want to do it like the goal is not to berate people. You know, the goal is to be educational, constructive and say, look, we like this. We like this. This part we don't like, but here's why we don't like it. And here's where you can work to improve upon it.

That a lot of what we're trying to do is give them, is make them better designers. Like the goal of anything in R&D is when you, part of being collaborative is the goal is to end up with the best product you can. And part of that is if you don't't like something or something isn't working, you're clear on what the issue is. It's not about people. It's never about you did a bad job. It's you made something, and here's why this is or isn't serving the purpose we need it to serve.

And if it isn't, here's what we're looking for. Here's why this is problematic. And we want a lot of communication. But for people that aren't used to that, that aren't in R&D, like I've been doing this a long time, and I have a pretty thick skin, but I mean, a lot of, is you do something, you make something, you sweat and tear, and you make something, and then someone goes, eh, I don't like it. And they go, okay, why don't you like it? And they explain why they don't like it, and that is part of the creative process. That's part of the collaborative process, is there might be things you really, really

like that other people just for some reason don't like and you have to understand why they don't like it and you have to understand what the criticism is so that you can adapt to it and improve it. Anyway, so we then send out the comments on Thursdays and then then, once again, now we're back in the loop. So Wednesday night, Thursday morning, I send out the letter to let them know that they've been eliminated, and then we send up the next challenge.

The idea eventually is the final three will come out to Wizards. My plan is to do, it's occasionally a podcast like this to fill you guys up on what's going on. So this is sort of a GDS3 in the middle. As I record this, I just handed over the second design challenge to Chris.

So, like, for example, tonight I have to send the rejection letter, you know, the letter to the person who got knocked out and let them know they got knocked out. And then tomorrow we will send them all the judging contents for comments for design test number two, and then we'll give them design test number three.

And the other thing behind the scenes is putting together design tests. There's some design tests of extra elements I have to put together. But anyway, that is how the GDS3 works. I hope, I hope, I hope by the time you're hearing this that it has started. The behind-the-scenes stuff is, there's some stuff we don't have a lot of control over. And it is, I mean, I would love for you guys to be seeing everything as we're doing it

and have it all be live and have everything go smoothly. It is just sadly not meant to be. But the good news is, I think when you guys all do see it, it'll be live, and have everything go smoothly. It is just sadly not meant to be. But the good news is, I think when you guys all do see it, it'll be more condensed, and I think that will make it a better show. Like, having to wait two weeks is not particularly fun, so I think we'll be able to condense it down a little bit for you once you see it. And I'm really impressed with the designers. I think the designers are doing really, really good work.

Like I said, what we are asking them to do is really hard. And these are people who do not do this for a living, that are not professionals. These are people who, you know, love magic and obviously would love to do it professionally. But this, so be aware that the whole process is a bit of a, it's a lot. There's a lot of work they have to do.

There's a lot of emotional ups and downs. And, you know, they turn in the design test and they have to wait to see whether they get kicked out or not. I know that's a very tense period. So it is, I want to, hats off to all our top eight. They really did a great job and continue to do a great job. Here's the interesting thing, by the way, my little halfway through. If you said to me right now,

who are the top three? Who are they? Who do you think they're going to be? I don't know. I mean, there's a few people that have been a little more consistent than the others. And so obviously they have a leg up right now. But it's interesting to watch

that most people so far out of the three weeks, I would say every bit of one person so far has had an off week. Like everybody's had a week where like, oh, you were in the bottom half this week and everybody minus one person has been in the bottom half every week so far over the first three weeks between the design challenge, between the design test, design challenge one, design challenge two. So who knows? between the design challenge, between the design test, design challenge one, design challenge two. So, who knows?

And we keep testing different skills. Each week, we are testing different things. And so, we might test a skill that one person

just shines at. Another person, it's their weak point. So, you know, that's one of the things that also makes it unknown is we keep testing

different skills and how they fare at the different scales. So anyway, if you've not yet turned in to watch the Great Designer Search, I'm hoping you can, please do. It is a lot of fun. And if you really want to see, if you enjoy how design works,

if you want to see behind the scenes, there is nothing more informative than watching people actually design cards and watching professional designers then critique and judge them. Every single card is judged by four different judges, all of which have, you know, significant experience making magic cards. So, and the other thing that's really neat you'll see from the judges is each judge has a different thing they're looking for because they have sort of a different vantage point. And so, you know, you really will get a chance to see, you know, the judges are all looking at the

cards in different ways. So you'll get a lot of different kinds of notes. Anyway, it's super informative. So if you like making cards, if that's the kind of thing that interests you and you somehow haven't heard yet about the Great Designer Search 3, check it out. I really think it's very entertaining. There's a lot of interesting to see. And there's a lot of drama. One of these people and maybe more than one, but one for sure, will get an internship and potentially a full-time job. One of these people could be leading the set that you're playing a couple years from now.

So who will it be? And when I say I don't know, I truly don't know. These guys are all, I think they're all guys. I'm not even 100% sure of that. We did not ask biological information of them, so I know very little about them. I know their names. But anyway, they're really good designers,

and I don't know what's going to happen. Any one of them could win. I truly believe that. So anyway, I'm now at work, so we all know what that means. This is the end of my drive to work. So instead of talking magic, it's time for me to be making magic. I'll see you next time. Bye-bye.