PurePerformance - 023 Is DevOps the Killer of traditional Performance Engineering?

Episode Date: December 5, 2016

Mark Tomlinson, “a veteran” in Performance Engineering, discusses how DevOps is a big opportunity for performance engineering – but also a threat for many that have been in the business for a long time. The big question is: are “traditional performance engineers” using their Load Runners or SilkPerformers at the end of the project lifecycle ready to change? Ready to learn new tools? Ready to think about automating performance engineering into the delivery pipeline and doing that in collaboration with the rest of the engineering team? Ready to “Check your Ego at the door”?Listen to our conversation where we also discuss how these roles have changed in organizations we recently interacted with.

It's time for Pure Performance. Get your stopwatches ready. It's time of Pure Performance. We have a very, very, very super awesome amazing show today. Although, as Andy points out, I say that kind of stuff about every show, right Andy? Exactly, you're a fortune teller. I know, it's amazing. Yes, and the reason i say we have a great show today is we have our good friend mark thomason back uh hello mark you are

a person a human being right you have um a lot of biological systems within you correct we're not talking to a computer is what i'm getting at right no that's this is this is not the computer simulated mark thomason excellent real in the flesh and uh mark mark is a uh a friend of andy and i's right we we go back quite a long time and uh you are you are our inspiration for starting this podcast and you have a you have a some a little small podcast of your own don't you actually i have some interesting news. The PerfBytes podcast is now branched into three. And Andy's joined Howard.

We're trying to figure out if we can get more of our performance engineering cafe going. And the News of the Dam podcast is taking off with James is taking that and run with it. And then starting in the new year, I'm actually thinking about doing my own new podcast called mark on task wow what about what about on the mark no it's actually it's a name of a painting that my wife martha did so i kind of named it after that you know i do have to say i do like her paintings i see every once in a while when they go up there yeah it's awesome if i had disposable income i would um purchase one yeah yeah so that that's coming a lot of this the podcasting stuff has taken off you guys are doing really well i'm really happy for you excellent um well i wanted to introduce you quickly because i figured you could probably chime in on this um andy well first

of all you're where are you at right now, Mark? I'm in Dallas. I'm in Dallas at STP Con, yeah. It's awesome. And Andy, you are at Java 1? Java 1 in San Francisco, exactly. Java R with the R in the end because I'm Bostonian now. What is that? Is that the Java runtime?

Java R. Yes, that is. Exactly. No, it's an interesting show. And obviously, you know, I think it's since Oracle took over and it's side by side with Oracle World, it feels Chababon is a small conference next to Oracle World, even though I think there's still like 5,000 people at the conference. It feels tiny compared to the 50,000 Oracle people in the city.

But it's fun. Good stuff happening. You know, it's very interesting because mark heard recently was on the record i think he was on one of the finance shows talking about how the fact that you you know with oracle you don't even need an it staff anymore it's just it's all in the cloud magically yeah so why is all those people are probably at oracle world looking for jobs? Well, I think, I mean, the cloud is very ominous here. It's like they are, I mean, if you walk over to the Oracle world part of the conference,

everything is in the cloud. It's the largest, fastest growing, best cloud in the world as they sell it here. It's attractive is there a golden escalator to get into the cloud i like that forget those beanstalks that's the old school yeah and you're staying at an air air b right there's no breakfast it's just air b exactly well uh so it's an interesting i had a very good experience last year taking an Airbnb. I had my own apartment. Everything was nice and cool and like a third of the price of a regular hotel because I just refused to pay $400 for a night, even though it's not me paying, but my company, but still.

So this time I thought I'd do the same, and I found an awesome apartment. And it's still awesome, but I share the apartment with three other software engineers who live here and they uh they actually camp out in the living air in the living room like with tents on the couches but it's really cool actually it's an it's a fun it's an interesting experience they i think they code during the day they play video games at night, and have a little different schedule during the day. So when I wake up in the morning, they're fast asleep. And when I actually am done with all my meetings with Europe and I actually get out at 9 o'clock here to my conference, they're still asleep.

But it's a fun experience. There's some cool guys. Is it all like Silicon Valley then? It's a little bit like Silicon Valley. It feels like Silicon Valley, yeah. That's awesome. So it is a good experience. And I definitely can encourage everybody,

if you want to not only save your company some money, even though maybe nobody cares about that fact, but it's a good experience. You meet people. The guy, one of the guys who is actually hosting me, he used to work for Uber. Now he's working for another startup. So it's very interesting chats that we have, very cool stuff. Mark, what do you want to talk about today, Mr. Mark, our guest?

You're awfully quiet. Let's rant about DevOps. Great. Rant about DevOps. Great. Rant about DevOps. Yeah, because I see sort of a generational or even a cultural split happening in the performance world over this funky thing called DevOps. And what is that you're saying? And you're on the, you obviously are on the old side of the generation. Well, thanks. But probably acting like the old man acting like a kid.

Do you wear a sideways baseball hat when you go to performance tests? Well, first of all, I will agree that some people need to dress their age. Because you see those old people, like, dressing like youngsters. And it's not, to me, that's just, you look silly. Is Hot Topic still a hip store for youngsters. Are they wearing like all the goth gear? And I don't know anything about that. I'm an old guy.

But no, I think to be honest, it's like anyone in technology. Let's just compared to other other disciplines or other industries. I mean, even technology, if you aren't keeping up on the latest, greatest stuff, like every three months, every six months, I mean, you're pretty much guaranteeing that you're going to be obsolete in your job in about a year. Even if you're on the same – we're just talking about updates in iOS. The whole game, the whole platform, everything can change what you thought was a valuable test technique or a valuable way to use a tool or maybe even getting stuck in a tool. You know, the idea that you stagnate happens in technology, you know, 10 times faster than any other industry. And so I think when we talk about,

you know, changes to tools or changes to platform technology, speed of computing and performance, obviously the CPU, the scale is gone off the charts in terms of memory, in terms of CPU, and now with SSDs and storage is really interesting. You know, you have to keep up. And so the idea of sort of being an old guy, quote unquote, an old, and like, well, I don't know what this DevOps is. Where's my gatekeeping? I get to stop a release, don't I? I get to raise my hand and stop a release.

I used to love doing that, though. Well, sure, there's a certain amount of power, and maybe your ego needs get met, you know, kind of thing. There's some other agendas happening, whereas I just see people who are like morphing into mob programming, into continuous delivery, and all these kinds of things. They have less, there's less in their culture, so to speak. They really do interact differently. They have different values. Those needs are not part of the equation.

It's like, hey, what can I do to contribute to everything we push out the door to make it better? And who cares what I'm getting out of it? Who cares if it's the tools that I'm stuck on? So to me, I'm not seeing like two or three or four different, quote unquote, camps. I see kind of classic old school waterfall style legacy tools and i see a bunch of new kids end of the age age aside right i'm just saying new techniques and new disciplines and especially devops kind of taken off it's a really interesting cultural experience to see these different groups i think so too and actually um what i see even i see them in an

even bigger extreme i would say and not only with our companies and clients that we talk to, also within Dynatrace. So what we actually did, and that's the same what I saw like last week I was in South America speaking with banks. that I see these companies that need to reinvent themselves, kind of breaking off and starting their own team that is working in the new way, call it the DevOps way, whatever you want to call it. They're building new teams from the ground up on new stack, new technology, new spirit, new culture, and then they let them run for a while to prove out that this is the best, this is the way we want to build software for the future, and totally disconnected from everything else the company's been doing.

I was even meeting with a client in South America, and I was in the morning, I was meeting with their, let's quote-unquote, old school, and the building was different, the people different it was just it was just old school yeah and these people had no clue about the news the new kids and then i went in the afternoon to the new kids new building glass palace uh everything open very cool you know like pair programming it was like i'm walking into a different world yeah yet it's the same company and these two teams don't even know about each other and really yeah because the reason why it was interesting so my girlfriend she used to work

in chile and she has a friend who now works for the quote-unquote old part of that company and i told him the night before i'm going to meet your innovation team and he said well what are you talking about there is no innovation team and i said well i'm going to meet your innovation team. And he said, well, what are you talking about? There is no innovation team. And I said, well, I'm going to this place. And he said, well, that's not a location that we have. And I said, well, let's see tomorrow. And then the next day I get there. Was the meeting at midnight?

No, it was not at midnight. No, it was really, yeah, it was not on platform nine, three quarters. No, it is really, it was amazing. And so, and the same thing just happened. So Gabi, my girlfriend, she works for Santander, for the Spanish bank. And I keep asking her, what is Santander doing innovation-wise? And she said, yeah, I'm not really sure. Over the last couple of months, I keep pinging her.

And yesterday, she sends me an email and says, guess what? We just got out of an all-hands meeting, and our central IT department in Spain just announced that they've been working on the Nextu platform for a while now, just basically figuring out on their own without telling anybody else, and now they're going to push it out into other areas. Yes, and then, of course, fire everyone else in the old building. Well, that's an interesting thing you say there, right? Because, Andy, if we go back, was that the, before I take it to where I was going to go, Andy,

were they planning on firing everybody or not? No, no, no, not at all. They just basically tried to figure out how are they going to build software in the next five to ten years? What are the new apps going to be built on? And obviously the transition is going to be interesting. There will still be the old, quote-unquote old folks, will have to keep the old apps alive, have to figure out how the new apps that are built cloud-native,

how they will interact with the old apps, because especially with banks, right, they have the mainframe systems that will still be there, so there have to be some interactions, some APIs. But over the course of the time, more and more of these older legacy apps will be ported or rebuilt, actually, I believe, with the new technology, with the new culture in mind. And I think either some people can make the switch over or they will have to find another job. Could be, right? Right. have to find another job. Could be, right? Andy, that's what a few episodes back

when we were talking with Adam from Capital One, it was a very similar story, right? They decided to go from monolith to microservice or whatever you want to call that whole process, just the whole continuous delivery. And they had QA teams, performance test teams, load test teams, developers, you know, all these other kind of legacy teams. And Mark, I'm saying this more for you and for anybody who hasn't listened yet,

but to that episode. But the difference that they had there is in order to make that work, which is where a lot of this issue comes up from, is they had a support from the top to say, we're going to switch over to this new way because we get it now. As a company, and I believe Adam said Capital One's new edict was, we're not a bank company, we're a software company now, right? So their whole concept there was, we're going to support you, we're going to give you what you need to make that transition. So if you're one of these older school people, we're going to help you learn the new way. You're really only going to find yourself out.

I mean, I'm putting words in his mouth at this point, but the general idea is you're only going to find yourself out of a job if you don't adapt. And you're going to have support to adopt this new process, though. So that, I think, is a key. So what I like with Capital One is that they actually give people the chance to level up, right? I mean, to actually learn the new ways of developing software. And most people, I think, take the chance. Some may are too resistant to change, and these are the ones that unfortunately will not have a future,

but at least not in their current job or in the new challenges. But I think overall the transition is coming. Some new teams, like with all the examples that I heard, are lucky enough to be on the new cool teams right from the beginning. And the more these teams show successes, I think the more of the legacy apps or traditional apps and the teams that support them will move more of the legacy apps or traditional apps. And the teams that support them will move over to the new DevOps-y way.

I think that's what's going to happen. I think some people will move over, Andy. To be honest, what I'm seeing is there are the, yeah, hey, I'm willing to pick up the new technique, pick up the new tool, pick up a new discipline, join a new team, you know, and pick up the new language. And I see that there's still, but there's some holdouts. And you'd think I would, after 25 years, be one of these old holdouts of the old way to do this stuff. But I'm not, just personality-wise, just for whatever reason. That's my new thing is like, hey, I'll help whatever context I can get in and be helpful and learn this stuff. And maybe it's, you know, there's, it's the opportunity in the new world, in the new building, um, with a new generation of software development techniques and, and performance. It's like, yeah, there's a whole lot of experience I

can bring, but I it's on, it's upon me incumbent on me to translate all of what I'm doing and all of my experience into this new language, this new culture, these new tools. And I think that's, to me, I still see there's some people who are like steadfast in the tried and true old school experience. And to me, some of those people, as I've seen them, sometimes that's a fast track right to the end of your career. And I don't want to see that happen to anybody because there are some of these guys that you and I know, we all know, these are some of the best experienced performance people in the industry, but it's really upon you as an individual or them as individuals to figure out how to take that performance knowledge and bring it into DevOps, bring it into new tools, bring it into these new cultures. And that can, with respect, that can be a real challenge, I think.

Yeah. And remember, Mark, the webinar we did, I think, was it earlier this year, where we, I think you had one of the areas of the webinar where you said check your ego at the door yes this was i believe this is actually the the best definition of the major benefit but also the major challenge for people adopting devops yeah it's really the you know you're not a lonely uh superhero fighter and you just make sure that you stay on your iron throne because you just do the stuff very well that you do. And you don't let anybody else in. And you don't share experience.

I think this is really the big change. Let me ask you guys, just since you're my brothers in the Dynatrace world. It's really interesting when you think about the Ruxit solutions in terms of people that are existing, elaborate, mature performance engineers using the main Dynatrace product. And there's these other options. And is there any parallel to the way even you guys see the Ruxit innovations working together, working side by side, or being maybe more of a fit for the new generation? What do you guys think of that? Well, I think the first thing we need to say is that we innovated Ruxit as basically trying to escape the innovation dilemma. We basically knew a couple of years ago if we want to stay relevant in the APM and the monitoring business, we need to build software for the next generation of applications

that need to be monitored that are cloud-native. That's the reason why we started the Ruxit project as its separate entity, basically, within our company. And what we are doing now, though, and if you go to dynatrace.com, right, we actually merged the Ruxit platform into Dynatrace. We also do this from a product perspective, that means what we want to be in the future, we want to be Dynatrace. That means we help you with your needs of monitoring applications that are either legacy applications that run on-premise, interacting with your old mainframe, but we also want to be the solution that you use for your new cloud natives

that are also talking maybe or maybe not with your old systems. And so we actually are combining these now, not only from a product perspective to help our clients, but also from an engineering perspective. We internally, we are now bringing the stuff that we learned from Ruxit across all the different products, but also the teams, the way we develop software. So in Ruxit, we had continuous integration, continuous delivery, and continuous operations right from the start. And a lot of this stuff that we learned there is now trickling also to the other product teams. teams um so i believe we innovated a separate lace in the beginning but then we we pulled it

into the other products where not only pull it in but they kind of came together so they moved towards each other yeah so this is you can see where i'm going like the parallel between the santander experience where you had sort of a very separate, innovative, you know, unencumbered by old standards, old processes. And let's be honest, we're human beings. We we kind of get into a groove. We do our job. Change can be hard. And so you you do start a separate team without the weight of that and or without being unencumbered by those things to try to innovate differently.

And maybe we're coming out of the curve in change because as Santander, you know, your example opens up to everyone. Hey, we've been doing this and you're you know, now you're invited. We're going to kind of bring it together. This is the same thing you guys are doing with Roxette and Dynatrace. And maybe there's other parallels. You know, we look at other acquisitions. Usually we see big companies doing acquisitions to pull these things together, and sometimes it doesn't always work out that way. I mean, the old IBM rational world was, hey, we're going to acquire this,

and then they never integrate. You know, it's funny. You talked about, well, first of all, I need to mention for everybody that you stole that Check Your Ego at the Door from We Are the World. Who was producing that? Bob Geldof. Quincy Jones. It was Quincy Jones, and then it was Bob Geldof was the Live Aid guy, right?

Right, but I think it was the Quincy Jones one that hung the Check Your Ego at the Door sign on. So I just need to call you out on that. Okay. But the interesting thing, you asked about Dynatrace, and Andy gave you a bit about the development labs and all that kind of stuff. Yeah. So I'm a sales engineer, right? And I'm in constant conversation with all the sales engineers, and we're talking about all these things, and we're looking at stuff. And we're going through that exact same transformation, right?

Because a lot of us who are traditional Dynatrace from, you know, I came in just before the original CompuWorkout acquisition. I'm diehard Dynatrace, what we call Atman now, right? And now we have Ruxit. And when Ruxit first started coming up, I'd look at it and be like, oh, yeah, that's pretty cool and all, but you can't do everything that Dynatrace does. You know, we go super, super deep and have all this great, amazing stuff in there. But as Ruxit's developing, we're also seeing Docker developing and all these other pieces developing.

And as we're sitting there, you know, a lot of us, you know, the traditional Dynastrace stalwarts saying like, oh, but this is the best product of the suite. You know, everything is great about this one. Everything is changing under our feet. And now we're all sitting there forced with that same exact situation of we have to start learning and understanding Docker. We have to at least be able to

deploy a basic Docker container this way we can have conversations about it. We actually now are like, okay, now we have to start looking at Ruxit seriously because this is all emerging together. And it's that mindset of saying, all right, I can no longer hold on to this old way. This new way is where it's going.

And, yeah, I feel that exact pressure. I will be out of a job if I don't learn these new things, if I don't start learning the product formerly known as Ruxit, if I don't start learning Docker. Even on myself, on my side projects, you know, I was in the past, I was always a load runner guy, right? So one of my side projects I'm doing to try to not only level up myself, but also just to have cooler things to be able to present and talk about is, you know, I need to go ahead and learn some Selenium. I need to learn a little bit of JMeter because one of the projects I want to do is do a fully automated in Jenkins. Like, hey, here's it's deploying a build. It's firing off some JMeter test scripts, and then it automatically then fires off some Selenium scripts and then spits out all the stuff at the end just to do that whole check in the build,

automate a bunch of tests and have it come up with the results at the end. But that's the kind of thing that if you're not going through and saying, hey, I'm going to make an attempt to learn that. And if I don't have the support from my superiors to learn that, you know, it's really hard. Yeah. Do we all, given that maybe we all agree that there is a transition from the old to the new and it's happening with like with technology companies, it's happening with big banks, it's happening culturally. I'm still, you know, to kind of bring it home for performance engineers, for me, it's like, what are what are the hurdles or resistance to this that I see being employed? And I'm always curious what what you guys think, like as you in the sales engineer, do you have there must be some SEs who are like kind of still having to check their ego or their dogma or their religion or, you know, not actual religion. I mean, you know, quote unquote, total religion or something at the door.

And so what do you guys think that resistance is? I think some of the resistance, besides some of those points you mentioned before, you know, ego, some of them, and this isn't to disparage any of the, you know, older folks in technology, but, you know, someone's looking and saying, I've got 10 years to get through before I retire, can I ride this out or can I go someplace maybe that has legacy still involved if this place ends? That might be a part. But I think one of the most confusing parts is,

all right, I have to learn new technology now in order to continue. But we know the technology market is so fickle, right? Everything is Docker today. Everything was Chef and Pop-Out. I know they're a little bit different and all, but everything changes. So how do you pick what to keep on top of? And, you know, that's the scary choice because I might start investing

and learning on something and then six months down the road, I mean, I'm sure at some point Docker is going to be gone. Something brand new that nobody thought of before is going to just completely wipe it out. So I think a lot of people have that defeatist mindset of, why am I going to learn this if in six or eight months it's going to be gone? By the time I know it, it's going to be gone. But guess what? A lot of people are still going to be on it before they transition to the new. Well, I think in general, in the end, it's about leaving your comfort zone,

right? If you have any experience with a tool and it worked well in the past, well, learn something new that you're comfortable with, right? I mean, you're not the expert anymore all of a sudden. And so that makes people uncomfortable. And then going through the rough path again of learning it again and becoming an expert again, that's obviously a rough path. And, you know, I think in general, obviously, this is true not for technology on its own, but for

everything that we do in life. Leaving the comfort zone is the hard thing, but we just have to do it. I think that's a lot of people who want to just phone in their job, right? You have, there are a lot of different people who work, right? Some of them are like, what do I have to do to meet the bare minimum requirements to get my check? And then you have, you know, and that's, I think, you know, what the leaving the comfort zone that Andy you're talking about entails. You're not going to leave your comfort zone if you have no desire to, if you're just,

if you're not passionate about your job and if you're not into what you're doing, you're never going to have that incentive or that motivation. And maybe it is best that you get left behind because this probably isn't the job you should be in. I know that sounds really cold, but maybe you'll get booted out of something, find a new career, and be really, really happy and really into what you're doing. Right. I think for some of the guys I talked to, and these are people at different customer accounts and things like that, that there's a particular breed where I don't know if it's so much like, I wouldn't say they're phoning it in. I don't think it's that perspective. I think it's kind of the opposite. Like they have sort of been in the golden years of being a performance guru at their company. And there was a lot of success and a lot of attention and a good feeling.

Like we're very proud of their work. And, you know, suddenly things start to change. And that good feedback or the kind of, I don't want to say charisma, but it is sort of the, hey, I'm, I feel really proud. And, and I don't want to say ego stroke. But yeah, you know, these people are really proud of what they've done. And the changes somehow put that experience aside, or even in some cases, I've met some of the younger generation kids that just don't even know to respect their experience. And it can be very emotionally difficult. But it's a real thing that organizations have to deal with and around performance in particular, because it's

this kind of special, it's always been like this dark art of, if you're a performance guru, you were usually a small percentage of the IT staff, and you're kind of a special guy, he or she, that person. And, and so I think that that may be part of the resistance, too, is I'm unwilling to stray from the thing that made me valuable and successful and proud of my work. And these new things, wow, well, now we share the accountability. I'm not just a specialist, or it's not just about me. I'm in a mob programming or a mob testing situation, or I'm in, you know, everything's being automated and I don't get to get my hands on everything all the time. So I wonder if there's just there, it's, it's the fear of missing out or the fear of losing

this, this good experience, the positive aspect of success. I think that's really a interesting topic, uh, idea you brought up there, Mark, because having gone through being a load tester to eventually running a load team, I started out in QA, moved on to performance. But always in my career, the performance of an organization looked at QA, you know, QA are just those people who go through and click and check boxes, whether something worked. And obviously, we know there's a lot more to QA than that, you know, the functional QA testing, but the there's that perception. And when you become accomplished and confident as a performance engineer, you start looking at your QA team members and say, hey, we're not, I don't want to be lumped into that perception as being part of QA.

And when you, as you said, when you get to that level where you get in the respect, you're getting the kudos, you're getting the feedback, you feel like you're finally making a mark for yourself. You're making a mark for the performance efforts in general. And then suddenly, when you go into this DevOps, agile, all these other kinds of components, you're just a wheel on the cog. Of course, you're all contributing, you're all learning, but you're no longer someone special. And in fact, in a lot of places that you're going to have to level up, you're at a great disadvantage again. So I think what you're saying there really could, I mean mean it resonated with me just just thinking about it like wow yeah you really do get put back in that position where

you kind of made a mark for yourself and now you're gonna yeah i have to learn some of these development things and some people some people some people might perceive that like a demotion i mean it may feel the same way and they're just gonna to feel inferior again you know yeah yeah anyhow so that's so the new old generation the innovation and then sort of merging i think that it always ebbs and flows and and waves and maybe we're we're right now uh seeing new you know agile has morphed into devops and continuousness maybe we're settling back down now where, hey, all right, we're going to make sure everybody gets on board who wants to get on board with this in a new way.

And if you don't want to get on board in a new way, well, you're right, Brian. Maybe you're in the last 10 years of your career, and maybe you go find another company that's keeping the mainframe running. Yeah. All right, well, let's wrap up this episode today, guys.

Mark. Yes. Thank you so very much for being on with us today, as we've told you before. And we do enjoy having some of the who's the man behind the curtain in the show. So as we did tell you before, we are trying to keep these to the 30-minute mark-ish.

So Mark, get it. Anyway, so let's wrap up today's episode. We have some more topics we're going to talk about. We're going to do a couple more shows. So I want to thank you for being our very special, lovely, wonderful guest. Very good. And don't let anyone know that I'm behind the curtain because I'm naked. Yes.

I don't think anybody wants to know that. I don't know. It's the joy of radio is no one will ever know. You know, under my clothes, I'm naked. Yeah. No, that's so accurate. Yes. So thanks, everybody, for listening.

You can follow Andy at GrabnerAndy on Twitter. I'm at EmperorWilson. And Mark, I don't remember yours off the top of my head. And it's changed. So it's now Mark underscore on underscore task. I'll have to add you to my new one. Mark on task. I think it still resolves the LeFord's M. Tomlins.

And I'm sure anybody who's listening to this who doesn't, or anybody who's listening to this probably already knows about PerfBytes. But if you don't, go check out Mark's legacy podcast. And we look forward to hearing your new podcast. When is that? When do you think you're gonna be starting that one up? It'll be January 2017. Okay, that's probably about the time this one will be airing. I'm kidding. Perfect. Thank you, guys.

Well, thank you. We'll be back with Mark in another episode soon. Thank you, everybody. Thank you, and goodbye.