a16z Podcast - a16z Podcast: Engineering Intent

Episode Date: August 30, 2017

"Young hungry and scrappy" is how Hamilton described his country, and it's how many -- including the guests on this episode -- describe startups... or more precisely, the mindset that engineers in startups need to balance both creativity and efficiency. But what happens as those startups scale, accrue technical debt, standardize their frameworks, and hire even more engineers? How do they deliver on their product while also staying on top of -- or better yet, using and also pushing forward -- new tech? (Even if that "new" tech is really the old, much-promised-before-but-finally-here, machine and deep learning?) And how do they do it all without getting mired in philosophical debates? Every Hamilton needs a Washington, after all...  VP of Engineering at Airbnb Mike Curtis and head of engineering at Pinterest Li Fan discuss all this and more (in conversation with Sonal Chokshi) in this episode of the a16z Podcast. The hallway-style conversation covers everything from taking an individual vs. company-wide view and the myth/reality of the "10x engineer", to the subtle nuances of how computers learn people's styles, intent, aspirations, and outcomes. And how all of this plays out as consumer tech increasingly connects the online to the offline world. ––– The views expressed here are those of the individual AH Capital Management, L.L.C. (“a16z”) personnel quoted and are not the views of a16z or its affiliates. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation. This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities or digital assets are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments and certain publicly traded cryptocurrencies/ digital assets for which the issuer has not provided permission for a16z to disclose publicly) is available at https://a16z.com/investments/. Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures for additional important information.

The content here is for informational purposes only, should not be taken as legal business, tax, or investment advice, or be used to evaluate any investment or security and is not directed at any investors or potential investors in any A16Z fund. For more details, please see A16Z.com slash disclosures. Hi, everyone. Welcome to the A6 and Z podcast. I'm Sonal. Today's episode is about the tech that comes into our lives in unexpected ways. But more specifically, we talk about the engineering that's hidden behind. and that drives consumer products that people use every day, like Airbnb, which people use to book homes, vacation rentals and experiences, and Pinterest, which people use to discover ideas or inspiration for things to try, from recipes and home decor to style and more. Joining us to have this conversation, we have Lee Fan, head of engineering at Pinterest, and Mike Curtis, VP of Engineering at Airbnb. In this hallway style conversation, we cover everything from what it takes to manage engineering teams

and the myth or the reality of the 10x engineer to recommendation systems, data, and machine learning to the camera as a vector for understanding intent, especially given the challenge of connecting the digital and the physical. So you both uniquely connect to the offline world, online to offline, and then back and forward, that's not just unidirectional. So a question I have for you is, how do you know what people are actually doing in the physical world? Isn't the biggest challenge that they essentially lose data when the person acts physically?

you don't have any freaking clue what they're doing. So how do you solve that problem technically? I think it is an interesting challenge. You know, the experience that you have offline is like your true engagement with the product, right? Yeah. Airbnb built really early on this review system, right? And an ability to be able to say like what kind of experience did you have and what happened. And one of the areas that we're experimenting with now is being able to use the reviews that we're given on a place as a predictive signal in the matching model all the way back. So we're able to feed. And why does a matching model matter? Meaning the prediction for like if you book this list, how likely are you to give it a good review?

So we can actually sort of collect some amount of offline data from the actual experience that you want to have and then use that as a feedback mechanism into our ranking model in terms of what listings we're going to show you next time. Because again, if like the objective function for ranking is really like how good of an experience you're going to have, then all this data you can get about what kind of experience did you actually have out there in the world can be then used to your next booking, but also to like be able to look at using that data towards how other people are going to book. Those reviews are not just reviews. They're much more. They're a vector

to more data for you to be able to do different things. In some way, we have a unique advantage that user, when they use Pinterest, a lot of them start with explicit action. They ping something they like. Those explicit signal user give us is unique assets, and this is a fascinating data set our engineer can work with so that we know one out of ten times. They're likely to go down deeper and a click by click. And sometimes when I'll say 20, they will buy something. You know, you guys are both talking about something so fascinating, which is a complexity is not just of search and matching and relevance,

but that a lot of people don't explicitly express with their intent. I'll have a George Nelson desk and a vintage French country bench. And those two categories do not go together. If you were to do an explicit search, one would do a mid-century modern and a different category for country or cottage chic. But now you have to infer this collection, in this cluster of traits of what people are interested in. Can you guys talk to me about some of the challenges of doing this?

First, difficulty or challenges, don't know what they don't know, right? For me, like, for example, very hard to describe. This is contemporary, classic, traditional. And also, you don't know whether this is the ultimate thing you want to match upon. Maybe you should put in the two different rooms, right? So I think one powerful thing we help user to do is, chances are some user have the same struggle with me. And the hard part is maybe that user is in Scotland and that they use different language to express. That's why lens can be powerful because you don't know how to describe.

Instead of ask a user to input a text as a query, take your camera, point to whatever you want to understand, and we will give you ideas, what are the related ideas. So you're actually saying that people, because people don't actually always have the language, and even when they do, there's different languages. There's a different A camera and image can actually level that. Exactly. And also you think you are looking for contemporary. Frankly speaking, you're one element of container.

You take a picture. We know, okay, maybe you like those styles. You think you know you're looking for this. Exactly. It's not necessary you have to describe in the English text. Right. You give me the image. I will tell you, like a thousand of the other users match your cabinet with this one

or match this sofa with the other table or something like that. You may not think your white sofa match is a real. red table, like outrageous. But when you see the image, you say, wow, that looks good. We want to give you that inspiration. In some way, this is different from Google. There's no right around. This is not the classic Google rabbit hole of you click on a bunch of page because you're interested in one topic. For me, it's fascinating because it's actually understanding yourself in a weird way. Computers augmenting humans. I'll give you a concrete kind of dumb example, but this literally happening in the end Pinterest in the early days. I had no idea. I love the

combination of dark, green, black, and gray. So whenever I did searches on shopping, sites. I would always put my favorite colors, but I never knew I loved that combo. And then I noticed one day that one of my Pinterest boards for dresses was all dark, green, gray, and black. And I was like, oh, my good, I love this combo. And in a weird way, the system kind of taught me what I like. I love that idea. And, like, the sort of act of, like, helping inspire somebody for what they might like that they don't necessarily know that they're looking for. I think is really fascinating. And it's, you know, it's actually kind of analogous to like the challenge that we have to solve on Airbnb because you think. How so? Well, I mean, you've got.

millions of travelers who are traveling, you've got millions of homes that they could potentially stay in. Every home is completely unique, right? Totally different. Like, we're not selling like a block of hotel rooms, right? There's so much variety in the style even. Every single one is unique. And so, you know, a similar problem for us is like, how can we look through, you know, like the click behavior of somebody who's going around and looking to stay in a certain area? If you're thinking about, you know, staying in Paris next weekend, you might have to sort through like 50,000 different places. Like obviously, we can narrow some of that down with filtering. But then we can also look at what are the similar.

and characteristics between the listings that you're clicking on or the ones that you're expressing interested in. And then the deeper you go into it, the more we can be re-ranking and surfacing other things that share some of those attributes, some of the areas that we're exploring now, is like, how can we, you know, find other embeddings in those images that can take the unstructured data of the image, turn it into something that can actually be tagged and labeled and then used in that ranking algorithm. Like maybe you particularly like places where you have views of trees from the window. And all the listings happen to have those Yeah. Actually, I do like that. I want that. I like that, too. Tell me how you technically

solve that challenge of moving from unstructured to structure data in that case. How do you now extract that data, feed it in? In the same way, like, you know, you can hold up your camera and it'll detect your face, right? There's... Like on Snapchat and Instagram and everything with filters. That kind of technology can be used to detect all kinds of different objects that could potentially be correlated with what would be interesting for you. And so it's that same kind of computer vision techniques that can be brought in to bring like that unstructured data forward and turn it into something you can actually use. And what does it take, though, to, like, move it?

What's the pipeline? First is you start with data. And I think both companies have massive, interesting data that user explicitly share their interest, right? You click or your book. The first step is really understand that data. We have computer vision technology to identify. There's a sofa.

There's a table. And we can train, label those, all those like millions, actually billions of things. We will have a subset of labeling data and say, this is a white sofa. And the computer will pick up all those labels. and start to learn. This is a white sofa. This is a leather sofa. This is a table with a brown color, things like that.

Once they learn, you will then connect your user data and say, okay, this user light to click, for example, the open loft feeling of house. And these are the things connection. Now you learn the connection. You say, we can do personalization for you. Now you go to, say, search the same query, for example, red shoe. We know if you are male, you are likely to want to have maybe a running shoe instead of a high-heel pump for women.

So those are the things we have to solve for personalize your favorite thing. And all those are powered a massive data set, the understanding of the image, understanding connection between image and a user, and use machine learning technology to make those connections. Okay, so you have, let's say you have users that are clicking on what they're interested in, they're browsing, you know, room listings, they're browsing pins.

How do you distinguish between the aspirational and actual outcomes? With both, you can have people who are pinning things because they want to have their dream house one day. or this is a type of Airbnb you're staying in because you can't afford to live in a beautiful industrial loft. But hey, when you go away, you can at least do it for a day or two. And that's a great way to get this experience. Do you actually weigh what they booked or actually pinned and bought more highly than the things that they might just be browsing and clicking through? Because one of the tricky things is differentiating intent when it's aspirational versus actual. And sometimes

you don't know is get an outcome that you can link to the choice that they've made. Because you might not know until weeks later that they bought that dress after. Oh, yeah. Exactly have the same problem. I would say it depends on user's state of mind. Sometimes users are ready to book or purchase. Sometimes they just explore, right? I plan my vacation months ahead and I start to explore. At that time, what I need is a creative, inspirational ideas. What's possibilities? That's how the mind works. Like, things sit in the back of your head for a while. At that time, I think what we could do as a product, expand your horizon and have, help you to discover new interesting ideas. We don't have to push you to into deep,

say, purchase this or book this. But then later, we can tell users' behavior, they're going narrow and narrow. Now I have this living rule. I do plan. Looks like you are going down and down. Where is this so far? I know I want to get leather. I want to have the recliner, things like that. Now we see those signals. We know, okay, you probably have the right intent. You're ready to do something more real. Then we will have. you to drill down. All those need a computer to understand the user intent and which state you are. Not everyone is ready to purchase right away. You know, similarly, when you think about differentiating between aspiration and an actual intent to do something, there's also another

dimension of like feelings. When you think about what in the early days of Facebook, you could only like or not like something. And now you can express a variety of things. We have emojis in a new communication form. And so we're essentially making emotions more machine readable. And And that's a really useful thing. But you don't get those signals, actually, because you're talking about people both in the physical world and maybe, I don't know if you guys actually have a range of emoji, not just a star for how you liked or not liked something or pinned or not pinned or not pinned something. How do you guys think about that's actually a lot of fastest to this? Could be like, you know, I'm looking for something that's a little bit more secluded and out of the way or I'm actually looking for something that's like in the middle of the nightlife. And like we can start picking up on those signals again based on like how you're searching and browsing through it. And then so that's sort of at the front end. But then. you know, all the way at the other end, we can look at, again, with that review information, there's some of it that is structured, right, like give a star rating. But then there's also the content of the review itself. So we can do sentiment analysis and, you know, some natural language processing on that to sort of suss out which aspects of this, like what feelings did

it evoke. You might have given it a five on cleanliness, but maybe you felt like, oh, it wasn't really like the right neighborhood. It lacked energy or didn't really have a warmth to it. Like, those are words that are interesting. People have such emotional connection. with these experiences that they have traveling. And I think, like, as a result of that, the reviews that people write and the comments that they have and everything are very rich and, like, filled with emotion about these experiences that they had. Don't you guys have this problem, which is typical with all review behavior, I think, where you

have this sampling from the extremes, you have this natural skew where only the people who are most extremely motivated because they loved it so much, or extremely frustrated, like, God, I hated that place. You regress to the mean when you sample from the extremes. How do you think about that? You know, there's such a personal connection when you're staying in somebody's home that I think there's sort of almost like a social contract a little bit. The percentage of trips that get reviewed are incredibly high.

By the way, you might have a unique benefit here too, which is that you have the boat size of your marketplace. Host can also be supply and supply can be demand in the sense that people who are hosts are guests and people who are guests can also be hosts. We get a high review rate from the host side as well. And I think that's also part of being a host and hosting, right? One thing we were worried about a while ago was that the reviews were really nice. You know, one of the reasons for that, of course, could be like fear of retribution, right?

Like if I reviewed you, the host and said, like, oh, yeah, this place wasn't that good. Then you say, well, this guest wasn't great either. How'd you address that? So one of the things that we did was we changed it so that we have a simultaneous reveal of the reviews. So both the host and the guest have two weeks to write a review and then they're revealed at the same time. So that way, you don't have to worry about like the retribution. They can review asynchronously, but the review is synchronous. So therefore, it's they don't know until they're, they know.

Right, exactly. And the reason that we went down that path, again, is also to make sure that we have good accuracy in the data. Because there's a lot of information in that review that can come through and be useful signal to us later on. I would say Pinterest is unique in that way because when we design, we really design an app or a product for user to feel positive. Yeah. Inspiration should be a positive thing. Yeah. In fact, when I use Pinterest, the main amazing thing, I figure every pin you see in like a so beautiful looking, right? And this is something actually surprising enough.

we are twin computer to learn. Why is this image of the same living room. You take a picture of this way and that way, they look so different. One just looks so inspirational, the other maybe just boring. And the computer will start to learn those cues. And they already learn certain colors. Yellow is more, you know, inspirational and happy than other. This reminds me of the old days of like color coding moods and color coding design.

But it's actually amazing because you can now actually do it in a way that's effective. Exactly. You can actually model. So what are the images? It's more inspirational than the other. Of course, it's not science. It's actually art. We learned, for example, if a same dress in a stock, very boring white background didn't sell.

We also want to add a personalization signal. We have some experiment in-house. If you take a picture of yourself and we learn your skin tone. But by the way, we can also learn from the pins you like. And we know there's certain style, certain shape of a model you like to. And usually it's people that are, it's similar that you relate to. So then then we will know those are type of dress or something you look for, right? It's not super fancy. For example, for me, it's more kind of a business like that I can wear. So I think all those, you know, signals we can pick up. And we also, I think one luxury we have is a user super engaged with our platform. So sometimes they're willing to give more signals. Take a picture of your dish. Take a picture of your living room. Take a picture of your kids, art. work. Just you think about the future where we're going with this connection between digital and to physical, there is a world of censification happening all around us, whereas more and more

sensors are embedded in our environment. You don't only have to rely on your smartphone to then do that. That example of colors is fascinating because you can see colors. You can see sofas, white leather, the style, this type of thing. But one of the funniest memes that makes its way on the internet. And I love this meme. And it's actually a fun new genre is, can computers tell the difference between a dog and a bagel? Or the latest one I saw was like Kentucky Fried Chicken and like these little like poodles that have the same kind of texture and a human can instantly spot the difference

and this is one of the challenges in deep learning because colors are easy but some of this is not. I would say if you give a classic picture of dog versus a classic picture of a bagel, the computer can solve at this point. However, if user generates random pictures with a dog background or with a dog in a weird shape

or whatever, it is hard. Remember, computer learns things from the data we train them, right? If you have millions of pictures of a dog and it with different shape, a different color, I'm sure it will get there. Right now, I will say a lot of domain is limited by the data. If you only have a limited data to teach, let's say, fashion, how can we know this is a fashion that are high end and more for the wrong way instead of a daily? It's a lot of data because it's a subtleness.

It's very subjective, everyone view differently. And then to have the computer really get there, it would take a while. I think the breakthrough recently in deep learning is that we can learn more sophisticated way. But it's a fascinating move so fast. Yeah, I mean, at this particular point in time and on the spectrum of a point of time, like, you know, a year from now, two years from now, or five years from now, if you look at like the advances that have been happening, just even really in the last couple of years in deep learning and AI technology, it is like an explosion over the 20 years before that. it's like these two huge forces sort of coming together and reaching a tipping point, the availability of massive, massive data sets, and then the processing power to be able to actually train models on them in a time reasonable way, like the broad availability of GPUs.

So I think what we're seeing is that we're at this moment of, you know, potential like exponential growth in this field that has been promised multiple times before in history, only to kind of not really have the ingredients there. The applications of this type of AI technology can go to every facet of how we live and work and, you know, how people sort of exist in the world. And what technology companies are really going to be great in the futures are the ones that think about this as this needs to be core to what we do. And we're not just utilizing this technology, but we're figuring out ways to push it forward. Natively. Because we're going to be half to, we're all going to have to be in leading positions in this technology in order to be competitive in the world that's coming in short order. I think that's so true. And I agree with you.

And let's shift gears and talk about that. So you guys essentially are at startups. You're still a startup technically, which is really weird to think about, given how big you are. And one of the biggest challenges is obviously getting to scale. And when you have that scale, you grow very fast. How do you think about balancing building this kind of competency as heads of engineering? When in a lot of traditional engineering jobs, I would imagine that you kind of know your roadmap already. And here in this kind of business, you shift direction.

How do you think about building this organizationally and operationally? How do you hire people? There's such high demand for people. who have skills around, you know, computer science, data, you know, AI machine learning. The technical challenge is so important and that has to be there. It's like the foundation for, as an engineer, am I going to be fulfilled by my work? But when you can satisfy that at multiple places, because, you know, we all have interesting technology challenges to solve. Then it starts coming down to like, what is the purpose behind my work? People more and more are choosing

their work based on the mission behind the work and the purpose of that work as opposed to like the specific thing that they're going to work on. Do you guys ever face, like, you know, there's a lot of these funny religious base with programming languages or, like, how you build your tech stack, like an open source versus like a non-open source and an own stack and a sheer stack or non-share stack? You know, open source, it can be kind of a lightning rod on some of these things. But I think we've basically just taken a position on it and said, like, you know, this is the way we're going to manage it.

We are strong believers in open source. We regularly open source our own internal technologies, particularly around data, like everything we do from managing data pipelines to how we do. and data analysis we pushed out to open source. And then internally, we kind of have the philosophy that if there's a great open source tool that can solve this problem that we're trying to solve, let's look to use it before we look to build our own. Because we want a higher percentage of the hours, like engineering hours and thought and creativity of work that happens here being focused on things that are very unique to our business. It's a progress way, right?

Because I think in my early days of career, like almost all the big internet companies do their owns that. And then we have tremendous progress in the open source community. Then you can say that principle without hurting your business efficiency. A lot of engineer I would say believe they can build better. And then it's a judgment call to encourage that or say let's look carefully. I totally agree when there is an open source tool that you can leverage. Let's pay attention and consider it. Sometimes I will smile when this question come up because I feel like engineer by nature they are so practical, but sometimes they go to the extreme of philosophical discussion. Engineers are secretly poets.

Yeah, and I'm like this, let's talk about this case, right? Instead of saying, are you in this or not? You're saying that instead of letting it be an abstract fight, you just go right down to the concrete of what you're actually working on. My strategy, when you go down the concrete scenes, and I always tell shared the content, share the information you have, what most likely happened is when you shared exactly the same information, your conclusion would be the same. But it takes a lot to foster.

to spend a time to communicate to each other and to spend time to understand, you know, there's not so big a difference that you're expected. We did a couple things relatively early on to try to head some of that off. The first is that we standardized our supported technical stacks and said, like, we're just going to operate within these stacks and not deviate from them except for an exceptional cases. And we also got, you know, actually a pretty broad engineering effort a couple of years ago to come up with, from an infrastructure standpoint, like, what are the tenants by which we want to do development?

And they're basically just guidelines. And the idea is, if you're an engineer, if you're working within the architectural tenants for how we develop software, then you're pretty much good to go and, like, just stay within it. If you want to go outside, here's the group of people that you can talk to. And I think that that has, like, sort of calm down a fair amount of that chatter. Don't get me wrong. You're like, I fight this battle too. Yeah, I think that's like definitely a good principle. It's just the constant debate that you probably agreed is really to encourage them to understand the benefit of a consistency and the company principle versus their own creativity, right? I think the engineer always won there. Like I have

you think about it individualistically by definition. Yeah, exactly. And you just have to constant balance. I want to ask you guys a question about technical debt. Big topic. You've had to grow very quickly and build things very fast and it means you leave this legacy of crap behind frankly. And I want to know how you guys think about that. I is firmly believe that we, you know, in some way, this is engineering excellence, right? You have to build a system or infrastructure that lasts long time, right, because it's ultimate efficiency. However, there will be a lot of product features we iterate, and we don't know whether

it will stick for three months, right? And maybe the features user don't want. Maybe something we will admit like, shit, it's a failure, right? So it's okay. So we want to be, I use Hamilton quote a lot, like recently, like, want to be scrappy. Oh, I love Hamilton. You know, want to be, you know, young, scrappy. Congress, let's move fast. At the same time, this is why we need a senior engineer

leaders in-house say, look, this is the system we plan, like I say, machine learning framework for the company. We plan to use for two, three years. That's exactly saying like you need a little Washington to balance out the Hamilton. It is definitely true, I would say. And also historically, you accumulate all those debt. And then let's face it. And I don't pretend it's there and not face it. We set aside time, say, fix it weak. I guess that's the only way you'll actually make things happen is if you carve out the time.

You just have to allocate, carve out time because otherwise you can say the principle, the engineer leads just want to shape in new things. This is human nature, right? You want to have fascinating new things coming out and look at that's like,

oh, system, like, but you need them. You don't want that the reliability will hurt you in the future. A great engineering leader finds the right balance like between time spent fixing things that came from before and doing forward development

because neither end of that spectrum is correct. Like if you spend all of your time having like a technically perfect system, you probably didn't do anything to further your business and vice versa. So I think like there definitely has to be a spectrum of time. The way we've been thinking about it and the way that we try to do it now is that the business leaders who are responsible for furthering the business also have goals that are associated with bugs, performance, like all the things like system signs business leaders.

Yeah. That's actually kind of counterintuitive for me, frankly. Yeah, I mean, because an engineering leader should be an owner of the business, right? thinking about like balancing these types of tradeoffs and the other angle that we think about technical debt on is really about what is our overall capacity as a technical organization to move like how much can we produce in any given amount of time and you have to be watching that because if technical debt crops up what happens is your overall throughput goes down and the worst thing that you can do at that point is say like well we'll solve it by hiring more people right

that's what people do yeah and like you can't hire your way out of that every every person that you hire just ends up becoming less and less productive and so you have to be able to look at like what is your contribution per engineer or per engineering hour work and seeing that that is like increasing over time through retirement of technical debt because then the organization has more throughput as a whole. One of our general partners, Martin Casado, wrote a post on why you have to hire VP of engineering early on and the big point he makes, because frankly, no offense to you guys, not knowing this world, I thought of it as roadmaps and deadlines and keeping track of schedules and you don't realize there's just big tradeoff between the big picture

versus individual pieces, even if they're empowered owners, by definition, a lot of individual people or groups don't own that big picture. And so how do you sort of connect those dots in this way and including making the decisions about priorities and tradeoffs? Because he gave this great example of how a way to accrue technical debt is that someone will actually go to an individual engineer and say, how long would it take to build out this feature for this customer? And of course, that that person will give them an estimate and say, well, I think it would take two weeks, not realizing that actually that's an estimate in isolation, not trading off the bigger picture. And you need to take into account who this engineer is. What do you mean by that? Meaning that engineer versus engineer, the throughput can be 10x or even 100x difference. Do you guys really believe in this?

I mean, is it a myth of the 10x engineer? I believe it. I also believe that we should reward according to. Oh, interesting. So really reward that throughput. I love that idea. I want to be a 10x editor. But it is a judgment call.

I think the leader to decide like whether there is a 10x or 2x difference. This is the beauty and why this field is so fascinating. a good engineer, the amount of the throughput or the contributing they can bring on table, sometimes it's beyond what you can measure or imagine. My thinking on that is slightly different in that. I think that people have periods of time where they can be like producing 10x what their peers might be. And it's that magical moment when they're aligned with the right project, with the right skill set, and like the right personal energy around it.

I love that you said that. Throughput is this connotation from the semiconductor industry. but it's actually bigger than efficiency because it's about the social, cultural context that actually supports this idea. And I love it because it reminds me of creativity. You have moments where you can do like 10 edits a day. And you have moments where you can maybe do one because you have a lot of meetings. Yeah, that's exactly how engineers and like everybody works, right?

Yeah. Like these moments where it comes together. And a motivated or not to make a difference. This is like one of the most core fundamental premises behind engineering management is like understanding the person, what motivates them, what are their skills, what are they trying to develop, that magic moment that I talked about, like being able to find those moments for people, and that being a core part of the responsibility that a manager has, because everything good happens then. I have to ask you guys this. You know, you both have founders and co-founders

who are not necessarily tech-quote native. Came from design product, but not classic tech. Is that sometimes a thing that you have to explain? Like, what's that like? I really enjoy it because Brian brings this incredible design sense. And it's not just to like how the pixels look on the screen. It's to like, how are things going to work? How does this ultimately pan out down the road. And I actually think that that in combination with the technical discussion would be super powerful. I would echo that. I actually really enjoy because they fascinated about the challenges and very supportive. When you have everyday conversation with a tech stack like people and you have formed a certain way because like implied this certain knowledge when you

talk to a designer, ask a question, it's like, wow, I never thought about this. And the process of thinking about how to answer this and discover there's some hole in my logic or discover like Maybe we were to, you know, go down this path and maybe we should think, in some way, encourage you to think out of box? I think this is a wave of the future, in fact, because there's been this traditional adversarial portrayal. And there's an idea that, like, engineers don't collaborate well with designers and designers don't always collaborate with engineers. And I think that's changed a lot in the last 10 years as you have a lot of these products, examples we're talking about today that they're touching lives, but there's deep, deep tech behind them as well. Well, thank you guys for joining the A6 and Z podcast. Thanks for having us.

Thank you.