Magic: The Gathering Drive to Work Podcast - Drive to Work #199 - Feedback

Episode Date: February 6, 2015

Mark talks about the importance of feedback.

I'm pulling out of the parking space. We all know what that means. It's time for another drive to work. I had to take my son to the orthodontist and into school today, so I'm starting from our school. But as long-time listeners know, our school is right by my house, so you should get a full podcast today. So I have a fun topic today. I'm going to talk about a very important tool that needs to get used. Feedback. tool that needs to get used. Feedback. So Malcolm Gladwell wrote a book called Outliers where they talked about how you get good at something

and you have to do it for 10,000 hours. But most people miss of that thing is it's 10,000 hours with constant feedback. Feedback is very important to the process. So I talk all the time about how what we do is iterative. And what that means is you do something, you get feedback, often from playtesting and stuff, learn, make changes, and then playtest again. And that part of the process, part of an iterative process, is using feedback as a means to improve what you are doing. And that feedback doesn't

always have to be playtesting. As we'll see today, there's multiple ways to get feedback. Okay, so what I've done today is I've broken the feedback into four sections. There's four kinds of feedback I'm going to talk about. First is self-feedback, how you are giving yourself your feedback. Next is team feedback, how the design team is giving you feedback. Then is department feedback, how other people in R&D are giving feedback. And finally, the audience, how the audience gives feedback. So I'm going to talk a little bit about how I think feedback can best be used and how it affects magic. Okay, number one, self-feedback. So I don't think a lot of people necessarily realize that you are giving yourself

feedback and that one of the most important things is understanding what you, what the feedback you have from yourself. What do you think about something? So when you playtest your game, what I recommend is that there's three questions you ask yourself. Question number one, is it fun? The thing I keep coming back to, I've had all podcasts on this, is at the end, the point of the game, it's a form of entertainment.

If the person playing the game is not enjoying themselves, you're fundamentally failing. It's not working as a game. Now, I'm not saying there can't be lots of other things going on, there can be mental stimulation, there can be all sorts of other things, but it is important that it is fun. And so when you are playtesting, one of the things you have to ask yourself as you're playing is,

is there fun here? Now note, early on in the playtest process, you are searching for the fun. It's not that the whole experience might be fun, but you're trying to find moments of fun. And as you progress, you're trying to make sure that you take those moments of fun

and you stretch them out and build your design around the fun parts. Okay, number two, is it accomplishing my goal or goals? And what I mean by that is, one of the major roles of a lead designer for a game is to make sure that they set the vision for what the game is trying to be. In my case, an expansion or whatever, I'm trying to set the vision. What are we trying to do? What is the goal of what we're trying to do? And one of the things when you

playtest is you have to ask yourself, okay, I have a vision. I have a goal. Is what I'm delivering meeting that goal? Is my game doing the thing that I say I want to be doing? Now, note, if the answer is no, there's two answers. Answer number one is, okay, how can I change my game to get the goal I want? And number two might be, oh, is my goal wrong? Do I have the wrong goal? And the reason you might come to that one is you might have a blast. The game is hilariously fun.

You're enjoying it. But you're like, oh, it's not doing what I said I wanted to do. But, oh, maybe that what it's doing is okay. Maybe my goal was wrong. That's something obviously that can happen. Okay, the third question you have to ask yourself is, can I remove anything? So I talk about this a lot too. A lot of making any creative process is figuring out what is necessary from what is not. I often talk about how when you write, you're always asking yourself, you know, do I need

this line? Do I need this scene? Do I need this subplot? Game design is you know, do I need this line? Do I need this scene? Do I need this subplot? Game design is similar is, do I need this rule? Do I need this element of the game? And that if you can chop something out and the game works well, chop it out. That nothing should be there that doesn't serve a purpose. Okay, so one more time, when you're checking with yourself, ask yourself, is it fun?

Ask if it's accomplishing your goals. Ask if you can remove anything. Those are the three main things you want to do when you are self-evaluating to figure out what is going on from playtesting and looking at your game. Okay, now we go a ring out to your team. So imagine we have design teams. So these are people working with you directly on making your game. Okay, so for each one of these I have three questions. A little structure for you. Okay, so the first question I have is, what is your team member seeing that you aren't seeing? And I mean

this in two different ways. One is, there's a lot going on in the game, especially in a Magic expansion. There's a lot going on. The reason you have a whole team playtesting is, there's just more people experiencing things. And A, they're experiencing things just because they're, like, you might play

blue-red and somebody else plays white-black. Well, they've seen the white-black cards and you haven't because you play blue-red. Also, each person has a different vantage point that they're coming from. So on our design teams, we have a developer, right? The developer's looking at, you know, is it balanced? Is this

stuff that we can later develop? The creative person's looking at, does it match our story? Are we conveying the essence of the world? You know, a lot of the core design teams that are on a design team are sort of like, is the design shiny? Is it doing what it needs to? Different people are looking

at your design in different ways, partly because they have different roles, partly because they have different roles, partly because they're different people. You know, different players might enjoy different things. One of the things you want to make sure when you have a mix for your design team that the different people have different desires. Partly is the role they play.

Partly is just, are they different kinds of players? Are there different things they enjoy about the game? You know, I talk all the time about how Magic in some levels is mini-games wrapped in one. You have different people that represent the different kinds of players. So the first question you're asking, basically, is, you can't see everything, partly because of vantage point, partly because of time, partly because of resources,

and that your team is filling out and being an extra set of eyes to see things. So first thing you want to say is, what are they seeing? You know, there's things you're not seeing that they are seeing. What are they seeing? And one of the role of having a whole design team is that you are allowed to get a lot of perspectives and just have more people looking at it. So number two, what is the group consensus? So another issue of having a team is to find out what does the team agree on? Where are people? Now, that doesn't mean, by the way, having the group agree on something, if it disagrees with you, the leader, doesn't necessarily mean that you need to change things, but it is important. Generally, what I find is any one person can believe whatever they want, but when a group starts believing something,

you have to sort of get the root of what they're seeing. Now, once again, that doesn't always mean that why they're seeing it or what's going on is correct at the time. It is okay to go, oh, my team sees something. I don't think that's the right thing. But you have to respect when your team all sees it, it means something. If nothing else, it means maybe the thing you're trying to do is not where the game is at because people are seeing it differently. Okay, the third question is, where do they disagree with you? One of the big challenging things about being the lead of a design is it is your job in

the end to fundamentally make the decisions. So let me explain this real quickly. I do not believe a good design is a democracy. The role of having a team is not so all of you together can make every decision. I mean, early on when I used to do magic design, I used to do more of that. Like, we would want to add cards, and I'd say, okay, let's vote on the cards. And I would try to get everybody involved so that all the decisions were reached by the group. And what I found was, A, the group does not have as clear,

like, you want the person who has the vision making the final call. That is you, the person leading the team. That doesn't mean you don't want input. That doesn't mean your team's input is invaluable. It doesn't mean sometimes your team is right and you are wrong. But it does mean that you need to be the one to make sure that everything is lining up with the vision you want. And so a design team is not a democracy, I believe.

I believe the design team is, they are a team made to help you, the lead designer, capture your vision. And part of your role as being the lead is making sure the vision is clear to all of them so they can help you with it. But you will have, one of the things as the person leading the set is, since you are the keeper of the vision, you will have a vantage point that is slightly different than the rest of your team.

And obviously, if you're doing a good job as a team member, you're trying to get your team to see what your vision is so that you get to a similar vantage point. But it is not about everybody sort of coming to a group consensus. You want to know if there is a consensus what it is, but that doesn't mean that the way you need to do the design is to get everybody to agree with you. Okay, the reason the disagreement is interesting is you want to understand where the conflicts

within your team lie, meaning if you're doing something and fellow team members think it's wrong, you want to understand it. So once again, one of the themes of today about feedback is one of the things that's most important about feedback is understanding the reasoning and rationale for why the person giving the feedback is feeling that way. One of the ongoing themes for today will be

that people are better at understanding problems than they are at solving problems. And what I mean by that is, when something is wrong, most people can tell you something is wrong. Now, not a lot of people necessarily can fix it, or the solutions they give might not be the ideal solutions, but they are good at recognizing problems. So when someone disagrees with you, understand why they disagree with you.

Now maybe when you dig down deep, they'll say, oh, well I believe thing X. And you go, I believe it's not thing X, I believe thing Y. I understand you believe thing X, but I believe thing Y. It's okay. Like once you dig deep and understand the reasons, you'll get to the core of what they believe, and maybe there's a fundamental disagreement. But a lot of times what I find with disagreements is they have an issue with something, and if I dig deep and understand what their issue is, a lot of time their issue and my issue don't have to contradict. The reason we might disagree is I or they might want something and the solution at hand is not addressing both our issues.

But it could. You know, a lot of times what I find is you get feedback from your team, they have a problem, and you're like, oh, the thing they care about I haven't been thinking about. The thing I care about I need to address. Oh, but there's a way to address the issue I care about and address their issue. And a lot of what I find the feedback from teammates are

is trying to get a sense of when something is slightly askew. And once again, this comes from the fact that they have different vantage points, and that is very valuable. Okay, let's get to the next one. The department. Okay, so in the department, what I mean is you're working in a team that's actively working on the project. The department are people that are with you but are not actually working on it.

So R&D in this case. So one of the things is when we do a set, there's a lot of people that poke in and take a peek. The development team obviously has to take a peek to make sure that things are developable the creative team needs to take a peek to make sure that we're matching their world and vision um the organized play has to peek in to see if we're doing anything that's going to impact uh how organized play works digital i take a peek in to see if we're doing anything that's going to be problematic for them to to you know um program um there's just lots and lots of, sometimes graphic arts will peak in

to see if we're trying to make new frames. There's all these different people that are poking in to get a sense of what is going on. And they're going to give you feedback. Okay, so first question is, what is their first impression? One of the things that's very valuable is

you are constantly evolving your game and one of the things that you lose is you don't have the ability after, you know, very early on to have any sense of the first impression of your game and that's really important. You want to make sure when people play that it creates a positive first impression. Well how do you monitor that when you're so ingrained in what you're doing that you don't have a fresh set of eyes? So one of the great things about outside people is every person who looks at your set for the first time gets to have an opinion, a first impression, that you can talk to them about and understand. In fact, one of the things I always do is when

new people come to R&D, the first week they have to get up to speed because they're behind by a couple years till they work ahead. And the question I always ask a new employee is, I want your first impressions. What do you think? Before I ask anything else, before I taint them with any other question, because any other question I ask them will kind of taint the purity of their first response. And the first thing I want to know is, what is your first impression? Now note, it's not important that everything has the most positive first impression. We do make mechanics that at first blush might not seem that good, but when you play them are. But overall, you want your sets to have good first impressions.

A lot of being successful is making people excited about your product. And that, well, I want the set to play well. I also want the set to have a good first impression. And what that means is you need to balance the number of mechanics that, like, you can have some things that don't look good at first, but turn out to be good, but you can't have a set full of those. If everything in your set looks, doesn't look good, your audience goes, I don't want to, I don't know. You know, they're, they goes, I don't want to, I don't know. They're not inclined to want to come play it.

And that what you want to do is make sure that your set has some exciting things in it. Okay, the next question is, what are they saying as an expert? So when people are poking their head in from outside, they represent something. They are a developer. They are a creative person. They are organized play, digital, caps, whatever they are. They're looking from the outside and they are... The one thing that's very interesting when you get outside is

the person who's coming from the outside has one distinctive vantage point and that it is a... You as the designer, are looking at a lot of different things. And it's very refreshing to get somebody from outside going, I just care about this one thing. Or these few things. And I'm going to give you comments on those things.

Because that is what I'm focused on. And my parallel here is, back in the day, I used to direct plays. And one of the things I learned about directing plays is that my actors have a vantage point on the character that I do not. And the reason is, as the director, I have to care about all the characters. I have to care about the motivation of everybody. But my actor who has a part, they don't care about anybody but themselves

because they're focused on understanding their character. So they spend a lot more time than I do focused on that one thing. And so a lot of times, if I want to understand a character, I'll sit down and talk with my actor to see what vantage point do they have. They might notice things that I would just never notice, because they're trying to understand their character, and they're looking at every line in the play as how it affects them, how their character interacts with everybody else. And so they have a very unique vantage point. The same thing is going on here, where

Eric Lauer, who's the head developer, he's looking at how to develop the set. He's looking at what he needs to do to be able to make a balanced limited environment and a balanced standard environment. He's looking at what he needs to do to make it a well-developed set. And so the feedback he's giving me

is not about the design, it's about the development. Same with the creative team. If the creative team is talking to me, they're focused on the story. They want the story to shine. And that they're looking at it to go, is this making the story shine? Does it make sense? Are the things that are incongruous with how the story is working? And so having

outside people is very valuable because the insight they are giving you is a very clean and clear insight that is different than the insight you're going to have. And once again, you will notice that one of the ongoing themes today is one of the reasons feedback is so important is people will give you feedback about things that you do not see or are not expert in or are not focusing on. And one of the things that's really interesting about other people is they will focus on different things

and they will give you feedback that is a very different style of feedback. Now, when we're talking departmentally, remember, they're the experts. Trust the experts. R&D is very talented. We have top you know, top-notch people working. So when I am dealing with somebody, what I want to say is,

okay, you're the expert. In the area, you're the expert. So for example, one of the things we do with design early on now, and we didn't always do this, is we go to the rules manager and to the templating team

when we have a mechanic that we think we're going to do. We spend the first, you know, early time in design figuring out what we want to do. But at some point, we're like, okay, yeah, yeah, we're going to do. We spend the first early time in design figuring out what we want to do, but at some point we're like, okay, yeah, yeah, we're going to do this mechanic. Once we know that, we then have to go to templating and the rules to say, okay, how does this work? How is it worded? We want to get a sense of what this actually looks like on a magic card.

And the thing that's important is I'm not good at templating. I'm not good at rules. I mean, I have a general sense of it. But when I go to them, they're going to say, oh, well, if you do this, blah, if you do that. And they're going to really spell out things that I might never have thought about. And that is why feedback is so important is as a person leaving the set, I do need to understand those things. They have a huge impact on what I'm doing. If I want people to enjoy my mechanics, I have to care about how it works in the rules. Is it intuitive?

How is it worded? Do people understand it when they read it? All that stuff's very important. Okay. The last question for the department is, what do they see that you don't? It's a similar question to what I asked for the team, which is, one of the things that's very interesting is, when you are working on a project and you're so close, they talk about can't see the forest or the trees. Sometimes what happens is when you have people that are less close to it, they can see things that are obvious but you're blind to. You know, one of the things that's very hard is

that each person is blind to their own faults to a certain extent. You know, that you can look at other people and you can easily see things they're doing wrong. But when you look at yourself, it's harder. That you have a vantage point that just makes, you know, when you're so close to something, sometimes it's hard to see things. And it takes somebody stepping back and going,

hey, this is happening. And in life, there's a good sense of feedback, by the way, about yourself, which is people will see you and be able to tell things about you that you do not see or understand. And people want to say, but hey, who knows me better than me? And the answer is, there's many things

about yourself that you know, but there are certain things that because of your closeness to yourself, because of your vantage point, you can't see. And there's value in having others to help you see those things. That is very much the same in game design. When you're so close to your game design, it's easy to not be able to see things.

Okay, the final feedback. The audience. So you make your game, it goes out, and your audience has some feedback. So one of the things that has been my role is, and this is very conscious on my part, one of the reasons I think I am a very good designer is I have made it my mission to create a conduit where I have an open communication between me and my audience. For example, I have a blog where I answer questions every day. You know, in the last four years, I think I've answered 45,000 questions.

I have a Twitter where I post to every day and I answer questions on my Twitter. I do Google Plus. I do Instagram. I mean, I try to be wherever I can be where an audience can have feedback and talk to me. And that feedback is very important. Okay, so what do I care about? What are the questions for the audience? Number one, what problems do they see? So, and this is another vantage point, but one of the things is, and I said this before, I will stress it again. People are good at recognizing there's a problem.

They aren't always as good as solving the problem. Usually what happens is people can tell something's wrong. They'll make a suggestion on how to fix it. Most of the time their suggestion is not the best suggestion on how to fix it. Only because if they're talking about your thing, you have more insight for how it works. Like one of the things that's very interesting is there are a lot of moving pieces in making a design work. The audience doesn't care about all those moving pieces. All they care about is the end result, which is fine. That's what they're supposed to do. The audience is like, am I enjoying this? Is this fun? They don't care about a lot of things that you care about.

Not that you shouldn't care about them, but it's not their issue. And that what they're trying to do is they'll explain why they have a problem. Now, their solution often won't work because they don't know all the issues you have to deal with. So they'll solve their problem and not realize it's causing a different problem. But recognizing what the problem is is important. Now, one of the things that's tricky is

Magic is a game that keeps breaking its own rules. Every time you break a rule, you will get people who are uncomfortable because people crave structure and rules. I talk a lot about just how humans behave. The human brain, I talk a lot about aesthetics and sort of like, the human brain loves structure.

It very much wants to understand things and group things, and it wants things to have a pattern to it. Humans also crave comfort, so they want to understand things. When you change things or make them uncomfortable, the first reaction usually is, what? You know, they get uncomfortable. Now, the fun of a game is it's in a safe setting where people can accept change. And obviously, Magic is a game about change.

But note that one of the things you will get, and every time I do something where I know I'm breaking from something we've done before, one of the things I do in my articles is I lay out rules. I'll say, here's our rules. And then when I break one of the rules, people write to me and go, what do you do? You broke your rule. And I'm like, well, we are a game that breaks its own rules. The question is, how and why do we break the rule?

And that one of my themes is, we don't break rules just to break the rules. We break the rule for a purpose and that we make sure that the rule breaking is done in a way that is safe and not hurting the integrity of the game. But anyway, your audience is very, very good at voicing. I mean, they're your audience, right?

They tell you what they like and don't like. Now, once again, magic is many games to many people just because, you know, one of the common feedback I give back for the feedback is sometimes people have a problem because the thing that they're upset for wasn't meant for them. And they're like, I don't like this card. And you have to understand whether or not it was meant for them or not. If it was meant for them and they don't like it, that's a huge deal. If it was not meant for them and they don't like it, well, you're going to take that with a grain of salt. If you're trying to make a really fun Timmy card and Spike's

like, I don't like it, that's horrible, I would never play that in a tournament, you're like, well, that wasn't for you. You know, and some of my answers when I talk with players is understanding when something was for you and when it's not for you. But, and this is important, no matter how, sometimes

people give feedback very negatively, sometimes they give it, they can be a bit cruel at times, they can be a bit cruel at times. They can be a bit harsh. Every bit of feedback, if you dig deep, there's a grain of truth in it. Something about it speaks some truth. Now, that doesn't necessarily mean each time that what they're saying, you necessarily need to change. It doesn't mean that you've made a mistake necessarily.

But it does mean that you're doing something that's causing discomfort. And you want to understand what that is. You know, why is the audience unhappy? Now, the audience will ask for things that they don't get. You know, the audience might want things that, okay, it's great that they want it. Either you secretly know that they wouldn't really like it if they got it or that, you know, it causes problems if you give it to them, or maybe you're planning to give it to them, but in two years and not now.

You know, there's lots of different reasons why they're not getting something, and there's reasons why you're not giving to them a purpose. But sometimes it's just like, oh, I didn't know you wanted that. Okay, we'll get to that one in a second. Okay, number two, what is making them happy? It is just as important to understand what is working as well as what is not working. The audience tends to want to give feedback on what is not working much more than they want to give feedback on what is working.

That said, the audience does give a lot of feedback on what is working. You need to listen carefully. When people have problems, they are very precise in what the problem is. They give you a lot of very specified detail. When people are happy, they tend to give broader feedback. And one of the things that's important when you have two-way addressability, when you're communicating with somebody, when they say, oh, it's awesome, you want to say, why? What did you like about it?

That positive feedback, you have to say, why? What did you like about it? That positive feedback, you have to draw out details a little more. Negative feedback implies them to give you details. Positive feedback often doesn't. And so one of the things you have to do with positive feedback is spend the time to get the information you need. Oh, I loved it. Why did you love it? What about it? What about this is different from other things? You know, what did we do here? So it goes into the third question. What is missing that you can grant

later? Or the better question actually is, how can you use the information now to improve what you do later? So there's two parts of it. One is, is something missing that you can learn that like

a lot of times you'll do something players want something you didn't deliver it but you're like oh okay I'm learning something why do they want

this thing that we didn't deliver and is that something that we can deliver later is that something that will teach us about things in general they want you know

when something is missing, one of the things that I always look at feedback is the point of feedback is the iterative process that I began with. That I want to do something, get feedback, learn from the feedback, make changes based on the feedback, and improve to continue the iterative process.

And I think of magic design as an ongoing iterative process. You know, I put a set out that's kind of like the equivalent of having a play test. I'm doing something. People give me feedback. I iterate on it. I put another thing out.

So putting sets out is a lot like having play tests, just on a broader scale. And one of the things, for example, I've been at this for 19 years now. One of the reasons I think that I've gotten pretty good is that I take that feedback. I listen to the audience. I talk with them. I hear what they have to say, and I make changes. Now, once again, one of the important things about feedback is figuring out where you think it's valuable feedback and where it is not. Not every feedback you address. Sometimes people will ask for things that in the end it's not good to give them. But you want to understand why.

And usually when I get feedback, my takeaway from any feedback, whether it is from any of these levels, myself, my team, my department, my audience, is there's something they want to tell me. I have to understand what that is. And if at first blush, I don't get it. If someone gives me feedback and I don't get it, I want to spend more time to understand it. Like I said, it's not important that you agree with every piece of feedback, but it's important that you understand every piece of feedback.

Why is the person saying the thing they're saying? What is the essence of their feedback? And the reason that is so crucial is the way you get better, the way your game improves, the way you improve as a game designer is that you take that feedback and you learn from it.

You want to get your 10,000 hours in with constant feedback. Okay, so the other thing that you can learn not only is what's missing, but also what was working. Like one of the big ways that you change things through feedback is you try things

and what you want to figure out is what are your successes and what are your failures? You know, it's very easy to focus on your failures and go, oh, I mean, when you make a mistake, it gnaws on you. I talked about this in one of my articles about how mistakes are great teachers.

Mistakes are awesome teachers. Success is not as much a teacher. The problem with successes is successes teach you to replicate what you've done, where mistakes force you to reexamine what you've done and try to find differences. So one of the things I always stress to people is, no matter what the feedback is, positive or negative, you want to walk away with, what can I do in the future that will address the feedback I've gotten? And the reason I stress it's really important to understand what people are saying when they give you feedback is,

you want to be able to turn that into actionable design. You want to say, okay, I've done something, I've listened to the audience, I got the feedback from the audience, this is what they're saying, okay, next time I do a design, I'm now going to design differently than I did before, because I have new data to work with. And I think if I look at, for example, I've been doing this design for 19 years, that I look back at a lot of old sets, and I made a lot of mistakes, but in a sense, I often think Tempest was my very first design. And in some ways, I look back at Tempest as kind of being like the Model T.

This early car that was like, in its day, was really impressive. It was this amazing thing. But you look back in the history of time, like, okay, we've come a lot farther. There's a lot of things I did in Tempest. I'm like, wow, if I had that to do again, I would do it differently. And it's not that it was wrong. Like, that's where we were at the time we made it.

And Tempest was very advanced for its day. It did a lot of things for the first time that had never been done before. That it was a stepping stone in us getting to where we are now, just as where we are now is a stepping stone of where we're going to be in the future. And that, to me, the value of feedback

is understanding that it is a tool in the arsenal of a designer. I mean, arsenal of any creative person, really, but I'm talking game design. That you, as a game designer, feedback allows you to help change the design as you're making it, and helps you change the design for the next time you do it.

Now, I happen to be in a product where we keep putting out things, and so Magic gets to constantly sort of innovate what it's doing, that we get to change, that we get to learn, and we get to iterate as we go along. And one of the reasons, by the way, I mean, I think a lot of people talk about how magic's really hit its stride, and one of the reasons for that is we've had a lot of time to try things

and figure out what works, you know, and we have a very talented group of individuals, and we have a lot of data. We've done a lot of sets and learned a lot from them, and we keep learning. Like, that's one of the things that, to me, is very interesting and why feedback is so important is there's no point in

which feedback is not valuable because there's no point in which you can't improve the product. I honestly believe that all that happens is as we learn how to do something better in the game, that we just find other areas that we can improve upon. Magic is a very, very complex game, not just from the game playing standpoint, but from the game building standpoint, from the game designing standpoint, that there's a lot of pieces moving and that one of the things, it's kind of funny when I talk about

the different stages of design, I feel like one of the things we've done over time is we've pulled back a lot, that early design was about thinking about how the cards worked, and then we talk about how the cards worked together, and how the mechanics worked, and how the sets worked, and how the blocks work together and how the mechanics work and how the sets work

and how the blocks work and how standard works and how larger, you know, that we keep sort of pulling back and figuring out how to do larger and larger things. And one of the things that someone who can look in the future is

we're getting more and more ingrained in how we do things. That there are questions I'm asking now as I do design that I never would have asked 10 years ago, 15 years ago, that I wouldn't have even had the mind thought to think of. There are advances to be made as we improve things.

We get to move our attention elsewhere, and we get to think in some bigger picture ways and more inter-set, inter-block, that we get to think long-term, and that to me is really cool. So anyway, I'm almost to work. So let me recap my lessons today of

feedback. So, number one, every piece of feedback is valuable. There's something in it. Make sure you dig deep and understand what that piece of information is from. You know, what they're saying. what is the core of a thing.

Number two is you don't need feedback. You do not have to act on every piece of feedback. Not every piece of feedback is necessarily actionable, but you want to understand each piece of feedback. And then once you have gotten feedback, you always need to ask yourself, how can I use this feedback to improve the quality of what I am doing? So, I hope as we walk away today, you realize that there is little more valuable to a creative person, a game designer, than feedback.

And that, notice today, I talked about feedback from yourself, feedback from your team, feedback from your department, feedback from your audience. All of that matters. There's lots of people from lots of vantage points. And that, remember, the reason feedback is so important is everybody else has a different vantage point of view. They are seeing things that you are not seeing.

And that part of being a leader of a design is setting the vision so everybody else understands what you're seeing and listening to what everybody else is so you understand what they are seeing. And the goal is to try to see what you are seeing and what they are seeing and line them up

so that you are addressing them correctly. And I think if you do that, I think if you use the feedback as a productive tool to improve what you're doing, that there is no tool probably more powerful than feedback. And my goal today is to sort of explain to you how to use it and how to listen to it. and the final point I'll make today is the reason I talk about not just the audience but your department and your team and yourself

is that the feedback comes at many different layers and that you need to be conscious of all of it especially by the way from yourself I think sometimes it's very easy to see feedback from external sources and sometimes easy to dismiss the internal feedback and what I've found is in some ways the most important feedback is the feedback you get from yourself, is understanding when you're happy and when you're not happy, and listening internally to your own sort of intuitive sense

of whether something is working or not. Because if everybody's happy but you, that also is something is still wrong, that you want to make sure that everybody's happy, you being part of everybody. Okay. That was quite... Oh, it's raining today, so we have some extra time.

Hopefully you guys enjoyed this today. I think this is a very important topic and something that really is applicable probably to more than just game design, although I'm talking about game design. But I'm now parked in my parking space, so we all know what that means. It means it's the end of my drive to work.

So it's time for me to stop talking magic and start making magic. So I'll talk to you guys soon.