The Changelog: Software Development, Open Source - Last DevRel standing (Friends)

Episode Date: July 12, 2024

Shawn "swyx" Wang is back to talk with us about the state of DevRel according to ZIRP (the Zero Interest Rate Phenomenon), the data that backs up the rise and fall of job openings, whether or not DevRel is dead or dying, speculation of the near-term arrival of AGI, AI Engineering as the last job standing, the innovation from Cognition with Devin as well as their mis-steps during Devin's launch, and what's to come in the next innovation round of AI.

Welcome to Changelog and Friends, a weekly talk show about DevRel and Zerp. A massive thank you to our friends over at Fly. Over 3 million apps have launched on Fly, including us, and youerp. A massive thank you to our friends over at Fly. Over 3 million apps have launched on Fly, including us, and you can too. Learn more at fly.io. Okay, let's do this. What's up, friends? I'm here with a new friend of mine, Jasmine Cassis, product manager at Century. She's been doing some amazing work. Her and her teams over many years being at Century, and her latest thing is just awesome. User feedback.

You can now enable a widget on the front end of your website powered by Sentry that captures user feedback. Jasmine, tell me about this feature. Well, I'm Jasmine. I am a product manager at Sentry and I'm approaching my three-year anniversary. So I've spent a lot of time here. I work on various different customer facing products. More recently, I've been focused on this user feedback widget feature, but I've also worked on session replay and our dashboards product with user feedback. I am particularly excited about that. We launched that a few weeks ago. Essentially, what it allows you to do is it makes it very easy to connect the developer to the end user, your customer. So you can immediately hear from your basically who you're building for, for your audience. And you can get basically have a good understanding of a wide range of bugs.

So Sentry automatically detects things like performance problems and exceptions. But there are other bugs that can happen on your website, such as broken links or a typo or permission problem. And that is where the user feedback widget comes in and it captures that additional 20% of bugs that may not be automatically captured. I think that's why it's so special and what takes it a step level above these other feedback tools and these support tools that you see is that when you get those feedback messages, they're connected to Sentry's rich debugging context and telemetry. Because often, I've seen it myself,

I read a lot of user feedback. Messages are cryptic. They're not descriptive enough to really understand the problem the user is facing. So what's great about user feedback is we connect it to our replay product, which essentially basically shows what the user was doing at that moment in time right before reporting

that bug. And we also connect it to things such as screenshots. So we created the capability for a user to upload a screenshot so they could highlight something specific on the page that they're referring to. So it kind of removes the guesswork for what exactly is this feedback submission or bug report referring to. Now, I don't know about you, but I have wanted something like this on the front end pretty much since forever. And the fact that it ties into session replay, ties into all your tracing, ties into all of the things that Sentry does to make you a better developer and to make your application more performant and amazing. It's just amazing. You can learn more by going to Sentry.io. That's S-E-N-T-R-Y.io. And when you get there, go to the product tab and click on user feedback. That will take you to the landing page for user feedback. Dive in, learn all you can. Use our code changelog to get a 100 bucks off a team plan for free. Now, what she didn't mention was that user feedback is given to everyone.

So if you have a Sentry account, you have user feedback. So go and use it. If you're already a user, go and get it on your front end. And if you're not a user, well, then, hey, use the code changelog. Get 100 bucks off a team plan for three-ish months, almost four months. Once again, Sentry.io. All right. We're here with our old friend, multi-time recurring guest.

Too many times to count. I don't know, Swix. Three, four, seven, 11 times on the pod. I'm not sure. But but you're back it's been a little while good to have you swix welcome back thanks good to be back uh been always a loyal listener and uh it's just an honor to be invited on every single time like it never gets old well we love your enthusiasm and your availability you know i i can hop on with you on monday and say, you want to come on the pod tomorrow? And you're like, let's rock and roll. So it just means I keep myself relatively free. It means I currently do not have a real job.

That's what it means. It means I don't have, I did move a meeting, but that's just because it's easy for me to move stuff because I am my own boss now, effectively. Remind me where you're at in the world again. Yes. So I am now no longer having a real job. I run, I always call it two and a half companies. It's the Latent Space podcast and newsletters,

the Media Empire, which you can talk about later, the AI Engineer Conference, which just finished two weeks ago. And I also have my own venture-backed company, Small AI, which I'm working on with a couple of engineers. So yeah, it's hard to describe because I don't work at a regular employer. I split my time between three business ventures. But that's just how my attention is spent. I think each of them independently have different time horizons of success. And hopefully, they all have a common theme of me being a prime mover among the engineering field.

How do you make money? So the thing that actually currently makes money is the conference that I run. We have now successfully run a 2,000 person conference for the first time ever. And same deal with most conferences. You sell tickets and sell sponsorships. You pre-commit to a whole bunch of expenses up front and then you freak out for three months hoping that you sell enough tickets to cover your costs. And then we do and we make some money back on top of it and that's the profit.

Well, happy to hear that you're running in the black there. A lot of folks run conferences in the red or very near the red. So that's awesome. Yeah, we had a lot of help because Andrej Karpathy basically tweeted about us and we immediately sold out like an hour later. So I think like for me, it's a long-term game of like, I want to build like the KubeCon equivalent,

like the industry, like the definitive industry conference for AI engineering, which is the thing that I decided to place all my chips on. So we don't have to talk about that this time, but I think it's relevant to DevRel in the sense that when I was a developer advocate or DevRel person, I spoke at conferences and a couple of times we actually even organized

company conferences. That's pretty much the peak of what you do as a DevRel. And I think for me, any DevRel who believes their own BS enough should actually go out on their own and do this as an independent business venture because it is worth much more to dozens and maybe hundreds of companies than it is to a single company. Yeah, that's interesting. Well, I remember last time you were on the pod, we had just experienced the chat GPT

moment, I think. It was probably a month or so after that. And you said, like, this is it. I'm going all in on this. I'm learning, right? You were writing what you do, which is one of the reasons why we invite you on, because you learn in public and we learn from you and with you. And you had made multiple transitions, you know, kind of front end stuff, dev rel, you

were thinking back end for a while. I know you were at Temporal doing workflows and back ends. And then it's like, all right, here's where I'm going to really dive in. And that was a while ago. So definitely want to catch up with you on that stuff. But the reason why you caught my eye this time around was a post about DevRel and about the zero interest rate phenomenon, which it seems like we're learning now post Zerp that a lot of things that we were living with and thinking they were normal, perhaps

were not so normal. They were kind of bubbly or frothy or maybe symptoms of or side effects of all of this cheap money, which was in our industry. And that quickly left our industry when the macroeconomic situation changed. You have a post which we covered in news just the other day, yesterday as we record, but a few days back as we ship, DevRel's death as zero interest rate phenomenon, where you ask and answer the same question, is DevRel dead? And I have heard a few whispers of this, like, okay, is DevRel dead or dying, or what's going on with DevRel as a thing? And so that's the opening of the can. Swix, take it where you want.

First of all, you say no, but maybe why is it not dead? I'll start there. I think to claim something is dead means there's no longer demand for a role like this, and that's objectively not true. I have friends who are desperately trying to hire DevRel with full knowledge of all its faults, because they are close friends of mine, and I have complained to them about the fail to hire DevRel with full knowledge of all its faults because they are close friends of mine and I have complained to them

about the failings of DevRel. So full knowledge that a lot of DevRel is completely ineffective. A lot of people are really bad at DevRel and they still have the job. Full knowledge of all that, they still need it. So you cannot say the role is dead if people just really demand it.

Right. But it is people, the fact, you know, just like with any technology, the moment people start asking, is it dead? It's not quite dead, but it's, you know,

it's less cool. It's not a good sign. Redux isn't dead, but people have been asking, is it dead? Yeah, you know, like it's declined and I tried to quantify it.

Right. I think maybe my approach is like the first time anyone's actually tried to say, it's not dead, but how much of it has died? And my number is 30%. Over Zerp, it increased 200%, and then it declined 30% since the peak.

And where do you get that? How do you quantify that? Google Trends. That most objective amount of data. A good proxy for perhaps being truthful, right? In terms of search, what's the trend there? Search trends, yeah. But also, that seems to coincidentally line up

with the common room industry survey of dev rels, where about 26% of them have said that they've been involved in layoffs. And anecdotally, we've seen a bunch of dev rel layoffs, including my old company, Netlify, including PlanetScale, including a bunch of other companies outoffs, including my old company, Netlify, including PlanetScale, including a bunch of other companies out there,

OffZero as well. And these are layoffs without replacements. So like straight up, we no longer have DevRel. And that is one form of DevRel being dead, right? Companies that used to heavily invest in DevRel. Actually, I'll put this on the record because I couldn't find an authoritative source. So I'll be the authoritative source. Microsoft, 2018 to 19,

hired something like 200 dev rels. They called them cloud developer advocates. Some of the top names in our industry spent really a lot of money gathering all of them. And like two years later,

half of them were gone. You know, it's like not polite to talk about power politics within Microsoft, but there was definitely a big power struggle in there trying to build a devRel org and not succeeding.

I think that was a really early precursor because that was in Zerf, right? 2018-19 was in Zerf. But it was a pretty early precursor and people having very inflated expectation of what DevRel could do for a business, throwing a whole bunch of money at it, and then realizing that just money alone and just the number of warm bodies alone doesn't actually solve it. You actually have to have taste and a clear message and a working system that scales healthily and effectively

and that takes time to build. There's a lot of facets to this, and one of which you mentioned maybe offhand that I think about a lot. I think Adam thinks about it a lot because we talk to a lot of DevRel's. Of course, DevRel's would love to be on our shows and all this kind of stuff. Right. And there's a fine line between a DevRel and a shill, you know, and like, I think there's

a big difference between good DevRel and bad DevRel, at least from where we're seated. Like we can just like see through certain things. And then other people were like, yeah, we'd love to have you on, you know? And it's like, there's a big difference between the two. It's, it's clear. And I wonder, we're like, do you think, first of all, you seem to agree with that? That's generally a true sentiment. Sure. Yeah, absolutely. And then, so secondly, it's like when it comes time to die, but then also have life, right? Like there's still value in the position. It seems like, and maybe this is just like stating the obvious or shallow, but it's like

the good ones are going to stick around and the bad ones are the ones, you know, no offense to any individuals, but the ones who aren't good at what they do, they weren't providing much value in the first place, right? Yeah. There's some amount of that, but also inherent in the job, it's a high burnout job, regardless of the macro, in the sense that it's mostly like a mid-career job. You know, there's very, very few chief DevRel officers. There's some CDXOs out there, but like there's no career path to like VP DevRel in most companies. Like it's just a, it's a job that you,

that you, that you sign on. A stepping stone. It's a stepping stone, or you've just decided to opt out of the rat race altogether, and all you want to do is make content. They're a lifestyle. It's a lifestyle business. And travel.

Yeah, well, for most people, professional travel actually starts becoming a drag. It's nice to see your friends every once a quarter or something, but professional travel actually is a drag like nobody actually really wants to do that in that role unless you actually just love travel but like that last like that's what my point is i've met a few and maybe they're just saying this but it's like they just love traveling yeah check back with them after like

two years of it yeah you know uh definitely yeah over time that gets old for sure you know there's only like even like you go to sedona or some special place for vacation like man i want to buy a house here which is how i felt when i went to sedona i was like man i want to live here but if i lived in sedona and i lived in orlando florida for a while too and it's like after a while orlando is just orlando now yeah i can go to Universal Studios anytime I want, but do I? I went there twice. Right.

Maybe three times over a three-year span. It's just not vacation places. It loses its luster. Exactly. It loses its luster in terms of like, well, I don't think this two years of travel constantly is really the game. Steve Klabnick, I think, is probably the game steve klabnick i think is probably the

best example that i've known of before you know well before even any of the zerp maybe it was even like maybe it was zerp i don't know it was for your money maybe that was part of it but not this phenomenon we're speaking of when it comes to like covet and like literally lots of free money out there this more recent occurrence of it steve klabnick traveled, I think, so much. And he was outspoken about this way, way back in the day and just got burnt, burnt out, bad on it. Right. Because it's just, you're not built for it.

Time zones, travel, you never know where you're at in the world. It's like you're constantly in a different time zone. Your body, your circadian rhythm can't even keep up with it. Yeah. So, you know, I want to preface this with, I think maybe some people's impression of DevRel is like it's a lot of travel. That number has definitely come down a lot.

Part of it is cost cutting. Part of it is environmental concerns. And part of it is people just don't want to travel that much. So actually, when I say people burn out of the job, it's actually not only that. It probably is not even majority travel. It's actually just the grind of constantly dealing with people

who are new to a technology. So I call this the eternal September effect, right? That there will always be more beginners. So you're actually, if you care about, for example, viewership numbers, you're always writing the one-on-one level intro to whatever. And that is your life

for as long as you want to do DevRel. Because that's the highest tab content that's possible, right? The best intro to something, that's the sort of pinnacle of your success is to write the best intro to something. So a lot of people don't want to do that forever. They want to have more seniority and more impact in their work. And impact being maybe financial impact rather than impact on the industry, because DevRel does have impact on the industry. And they choose

to move on. So people leave DevRel not only because they're not good at their jobs, or they just aspire to something different. And that's normal as well. This job, way more than other jobs and startups, has more churn inherently and that that's that's normal i just want to establish that like it's not a judgment thing of like oh you you sucked at it so it's like you know you had to leave yeah so so like this was a very overall this was a very tough piece to write because obviously a lot of my my friends are devrel like i obviously i had that job for for a long time and a lot of people know me for that

and come to me for advice on that. And actually, like, so this post, the idea for this post was actually one calendar year ago.

I tweeted it on my sort of private alt account and I had to wait for everyone, for more people to start saying it for me to be okay

publishing what you could read today because me saying it too early would have pissed off a lot of my friends who had that job and and like i think for me to find the words that would accurately try to say what i was thinking without also like being too inflammatory right like i'm trying not to bite the hand that feeds me, but I'm also saying like, hey, like let's call a spade a spade. Like there was a mania in DevRel over this Zerf period and now everyone can see it.

Let's put an end to it. Let's try to learn some lessons from it now that it's okay to say it out loud. What's up, friends? I'm here with Michael Abbott and staff software developer at 1Password. Michael, I want to talk about how 1Password helps secrets management, secret sharing with teams, with orgs, with production, even really easy, even so much so that when you spin up a new person onto a team or you add new people to a team or you even if you take away people from a team, you know, you want to focus on access and you also want to focus on security. So if you want to remove somebody, you want to make sure they no longer have access to your secrets. And then

the flip side is when you add somebody to a team, you want to make sure they have access to their secrets. So help me understand how one password helps teams be efficient with secrets management. So when you have a dev team storing their secrets in 1Password, then it makes it really easy to bring new people on. They already have so much to do and so many new things to learn. You don't need them to have that extra burden of creating a dozen different accounts to be able to access your payments or your errors or your monitoring. You can have that all set up for them already within

1Password. And each of those particular services gets pulled into your local application, your production application at runtime through the 1Password CLI. From there, you have your new developer downloading the repo. They don't have to spend time setting up their environment. All of the different services are ready to go because they're already stored in 1Password. All they have to do is use the 1Password CLI to spin up and run the application and it's ready to go. It's like developing in the future. Well, we must be in the future, Michael, because we use 1Password just like that. All of our team secrets, all of our personal secrets, all of our application secrets, they're all in 1Password and we're using exactly this process to make our lives easier.

So friends, go to 1Password.com slash changelogpod. They've given our listeners an exclusive extended free trial to any 1Password plan for 28 days. It should be 14 days, but no, it's 28 days, but make sure you go to one password.com slash change all pod to get that exclusive signup bonus or head to developer.onepassword.com. Of course, to learn about one passwords, amazing developer tooling,

the one password CLI, one password for SSH and get CICD integrations, service accounts, and so much more. Once again, 1Password.com slash changelogpod. This Microsoft area you mentioned that was pre-Zerp, and maybe even after it, like I think one of the reasons maybe there's churn is that it's a hard, even to this day, kind of hard to define what exactly is dev rel. And when you have a challenge defining what it is,

you have a challenge in defining what it should do, what the function should do, which maybe is the reason why people flunk out of the job or move along or churn, because maybe even in the Microsoft era that you referenced where they hired lots, it was challenging to define what exactly are you trying to do? Because if the job primarily is focused on a function that sits between the company, which usually is a tech product of sorts, a SaaS, a dev tool, a dev service, etc. Maybe you have an open source company, a cause company, or an open source project, they can have dev roles as well.

If the function is to nurture the relationship between the company, potentially the purchase of a product, and the developer community, there's a lot of ambiguity in there in terms of what you can do to be successful and it might be challenging even even as a manager to manage dev roles right like that's like what do you really do here what can you do here what is success in your role and when you have lack of clarity as an individual it's kind of hard to maintain what a good friend of mine michael burt coach michael burt says the prey drive right you have to have a a reason. It's your because goals. Because I'm a dev role, these are the things I do in my role.

Or because I want to do these things within the developer community, I have clarity. When you have ambiguity in your role or lack of clarity, it's kind of hard to kind of wake up every day and be motivated and get something done. Or when you're tweeting or doing social media or doing these things that aren't really seen by peers or adjacent peers like engineers or marketing or sales, it's like, that person is Devereaux and they're just tweeting. Like, is that work? So when you feel like you don't really, I mean, like when you're not clear, it's kind of hard to kind of just get up every day and just do what you do well, unless you're a self-motivated person. So I guess all that to say, how much of this churn is because the management or definition of what the role is

and what success is of the role is well-defined so they can be successful? I would say, so I worked at AWS, says Deverell. I've never worked at Microsoft. But I do think you should have some faith in the big corps to really define roles before they hire for them, because it's hard to open headcount in these things.

And so they have their definition. And I think there's more security there just because these things are very, very well defined, at least internally, for that stuff. What's less defined is the startup side of things, where I just got a bunch of funding. I'm going to allocate one person out of my 15-person team to go be that public face for my company. What is your job? Do whatever seems right. And I had that job. That was me at Netlify. I had no manager for a year. It was fantastic. It was also absolutely Zerp because I just did whatever I felt like

doing. It was fantastic. And then eventually we got adult supervision with Sarah Jasner. But also I thrived. It was the most intrinsic motivational job I had just figuring out the metagame. And I think a part of it is, with marketing, with anything, with people, the true Tao is a Tao that cannot be taught. And the true DevRel is not the DevRel that can be written down.

And the moment you try to write it down and try to systematize it, the game has already moved, you know, six months ago to like the new game, right? If you think like, here's the way to success and we're going to scale this

for the next five years, tough luck, like people move, people in trends and, you know, move quicker than that. So it can be a really tough thing to nail down. That said, I do have a piece that is fairly popular. I'm told it's like required reading within Google of measuring

DevRel. And I think basically, you know, the definition of any job, you know, just from the outset is that so you treat DevRel as a black box, money goes in, what comes out of it? So I have three major buckets. It's community, it's content, it's product. And either you're producing in one of these areas or you're not at all. And I think most expectations of DevRel is one of these three things. And I can go into those things further, but you should have some definitions of what the visible output of DevRel should be from there. And then that basically becomes the job. And if it's not successfully captured in those three buckets, then you're

probably doing a different job than what most people think of as their role. I think that's well said. I think that it is really tricky because almost the more formal and described and delineated the role becomes, you're probably on the lower end of the value chain in terms of actually being able to execute on it well for the reasons that you just stated. You almost need this, the formula that works really well is like the small startup.

And I think you talk about there, there are companies who are doing really well with no dev rel or minimal dev rel, but they all have like a charismatic leader or somebody who's already very online and very good at being online that just continually brings them more and more interest, more and more community, more and more relationships. And that person's almost a unicorn to a certain extent.

Like can you systematize what they do and then hand it to somebody else and say, go and do this? I'm a bit skeptical. Maybe it can work, but I don't think it's going to work on a repeated basis into the hundreds of employees, right? Into the hundreds of employees on a smaller scale, yes. If we're talking about like the Twilio's of the world, which I feel like they have successfully done that, that is, it becomes much less personality driven and much more about a repeatable process that can be scaled across every major city. And it might be also a relic of maybe seven, eight years ago when things were maybe more, more,

more sort of in-person centric. Now that I think online has, has taken more and more and more and more of our mind share and time and, you know, remote working and all that, the sort of online first DevRel, meaning that it is just no location, means that there's just no need for repetition and therefore more centralization in a single person.

And so I think that's important to note. I think the other thing also, like, so I agree with this whole, you know, the sort of low level junior beginner person thinks about it as like, oh, I will produce three blog posts a month or something. The higher level thought process is I am persecuting a mission. I'm promoting an idea and the output is three blog posts. But those are downstream of me promoting the idea. I'm starting a movement. For me, when I talk about the stuff that I do for Latent Space, AI Engineer Conference, and Small AI, I don't think about it as I organize a conference.

I think about it as I am starting my own industry, and the visible output is I start a conference. And so I think people who think about it on a higher level have a more coordinated approach to their actions, even though the actions still look the same, they have a more cohesive outcome because there's a broader plan beyond that instead of the individual units. And I do think for the junior dev roles, I do some advising on the side just for fun. And for the beginners who come into the job, they very much do the one-off, like, hey, we got to get this launch right. We got to max out this launch and we got to get all the retweets,

make this the most awesome launch possible. Whereas for me, it's not about the launch. It's about the long running campaign from past the launch through whatever's next. And I think getting people to see that whole journey, I think is something that I think levels them up to the next tier. And I think that's what see that whole journey, I think is something that I think levels them up to the next tier. And I think that's what the founders can uniquely do.

And you talk about these unicorn founders. That's what the founders can uniquely do apart from the hired guns, which is the founders started the company because of this whole mission and being able to authentically tell that story, I think is very rare. I think maybe one possible positive example

is Lee Rob at Vercel. It was successfully taken on that sort of voice of Next.js. A great DevRel, by the way. When I was thinking about the good ones, he was the one that was in my head. It's like, I think the guy does a great job. He actually prompted this post

on the DevRel because he was like, yeah, like I've been thinking about this a lot because he and I chat offline quite a bit. But yeah, I mean, you know, like it used to be Guillermo primarily promoting Next.js and Vercel. And now I think Lee Rob is kind of number two in the company, at least as public facing figures go, being the voice of the company. And I think that's absolutely a rare success story. I think it's very much a two-way street about the founder

being able to trust whoever they hire and then the employee being able to rise up to the task. And very often those two things don't happen in either direction. So that's what it is. The last thing I want to point out as well, just in terms of why people fail at the job or why measurements fail. This is something I've been thinking about, which I did not sufficiently capture in my measuring rel post, which is that often DevRel, just like any other content or media business, is a hits-driven business, not a consistent output business. I used to often say, I'll write 50 blog posts a year, but just one of the 50 will actually be the one that people remember. And it's very, very hard to run any business. It's very hard to learn from any information

or any market reaction when most of the stuff you work on is going to flop. And it doesn't mean that anything is wrong. It means you should still keep going even though most of your stuff is flopping because it's the stuff that's going to produce that one hit and it's going to justify everything else. Right.

No, I 100% agree. Any hits-based business is like that. Well, you say, okay, well, all I need is hits then. I'm just going to only publish hits. It's like, yeah, exactly. Well, see, that doesn't... You don't know why it's... I mean, even with this post,

Swix, you said, I'm not sure why you guys invited me on or I'm not sure why this one resonated with people there's many reasons why i'm not sure i don't even know why but i was like oh this is interesting and uh hadn't talked to you in a while i could probably answer that question okay well i think that we talked to a lot of dev rels we have a lot of friends and fans and people we're fond of that are in this space that have i would just say tumultuous times right and so it's it's a hit because we don't want to see people or companies shrink that particular function size because that means friends of ours are out of work or they're changing what they do,

they're moving into adjacent roles or different roles, maybe directly into marketing where DevRel is sort of adjacent to it, but sort of in a lot of cases under it, sometimes even under product. I think we care because it shows the health to some degree of our industry. If one of the core functions is withering or failing or churning or not right, I think it's an indicator of how healthy the market might be. I think that's why we care about this particular function so much because this is literally where where product and the future potential buyer might be. Like one thing you reference in your measurement is the Sean Ellis question, which essentially is, how would you feel if you can no longer use the product? And so if you ask

this question to a free user, a free tier user, you say, how would you feel if changelog was no longer a thing tomorrow? Would you be happy, unhappy, somewhat disappointed? Devastated. Very disappointed. Devastated. Thank you, Swix. I think that if we had a large majority of people saying devastated or very disappointed versus the other two or three options, then that means that we've

got some version of product market fit or we're very beloved. And so we should find a way to exist or live if that were us on a deathbed, Jared. Jeez, this is terrible. Point is, is that I think that we have a lot of people who are in that space we care about. And any unhealthy measure in this particular space shows signs of an unhealthy market. That's why we care. That's why I care. Plus we're looking for answers and explanations too, right? I mean, we see things going on.

Some of us talk about it. Some of us don't. And it's tumultuous and it's scary and sad. And then you're looking for answers. You're looking for like, well, what was going on? And it's like, well, here's a post that surmises that it was this. And now, okay, that rings true. I mean, it rang true with me, which is why I put it on ChangeLog News. And then I was like, and I liked the end of it. Like, well, given all of this, like what now? Or what can we actually

move forward? Because we know that it's not dead in so far as it's not a valueless thing. There's huge value in having high quality developer relations around your product or service and all that that entails. But the free money is gone, which was allowing it to bubble. And as we've discussed, now what? In that post Zerp environment, what does it mean for dev rails what does it mean for everybody else is it still a job that i should go out and seek you know is it not is it something that i you know like what are the now what so switch key in on that point and what are some of your thoughts being deep in this area of here we are 2024 halfway done who knows what's going to happen by the end of the year,

but I don't think we're going to get back to zero interest rates by then. We might see one, maybe two cuts from the Fed this year. Maybe not, maybe zero. But for folks who are either in DevRel currently or considering it or trying to get back into it, what are your thoughts for them? Yeah, so I actually tried to leave solutions out of this post because it's been covered elsewhere

and I haven't identified as DevRel for maybe a couple of years now. And there's a bunch of solutions out there. I do think just the straight job hasn't really changed. I think what the removal of free money has led to is basically we can no longer get by with lack of accountability in DevRel. It's probably a good thing.

It's probably something that we needed. And so what I tried to do in the post is to list out the smells of what Zerk DevRel looks like. And so I tried to use that as a checklist for people in the industry of like, if you were doing this, there is no longer any appetite for this. Like it is no longer okay to do like, let's just call it

a free tier DevRel, for example. Like you only talk about how to use your company's free services and have blissfully zero knowledge of anything paid because that, you know, that doesn't serve the company's needs. And also actually, you know, more to the point, it doesn't actually serve the customer very well because you don't know your product. And also actually, you know, more to the point, it doesn't actually serve the customer very well because you don't know your product. And there was a lot of free tier dev rel in Zerf because it's easy to talk about something that you can adopt for free.

It's easy to get applause for something that's free. The hard part of your, and what's really challenging your skills is saying like why, you know, your company's products are actually worth real money. And people who obviously were successful at that

were probably more valuable to the business anyway. So yeah, there's a lot of thoughts. So at the end of the post, I linked to Lee Rob, I linked to Sam Julian, who used to be VP DevRel at OffZero, and I linked to myself as independent thoughts. I think everyone's basically, the common consensus, let's just say, is that DevRel moves into developer experience, which is kind of an annoying rebrand that every industry likes to

rebrand itself. In DevOps, there's this ongoing rebrand to platform engineering. Same thing for DevRel. Same thing for data science, right, Specs? Data science? We can talk about that after, but I would argue not. Okay, save it. We can save that. For me, I do think that basically there's a maturation DevRel that I'm

looking for where you don't have the one-size-fits-all DevRel that does the full stack of production to publication to idea generation and you have a front- middle, back office DevRel. This is definitely for more scaled up organizations. I was leading a team of nine at my previous company.

And I definitely saw that need to grow more structured process around DevRel. And then I think understanding, for people who choose the developer experience path, understanding how you interact with the rest of product and engineering and having the buy-in to actually have features like for, for Lee,

like he straight up just became VP products. Like there was no, we will coexist with products. No, like DevRel just took over product. Like that's, that's how they solved it. Very,

very few other companies will actually let that happen because products usually has way more political power than DevRel. That's just how it is. And so DevRel then gets shunted to marketing and then loses all power from there. I think, I think like having DevRel become PMs is the path that like, I think I see the, some of the really motivated people interested in impact do. And I think, I think that is think that is the right way to do things. But DevRel as a title is going to continue to exist as a primarily sort of marketing and community and docs and function much more than product,

just because product is its own beast. It's a much more established industry by far and much more politically powerful and therefore a harder force to have any impact on. I don't know if anything I said is controversial. Well, leading product is tough. That's a tough role. What do you know about how things have changed for Lee Robb? Because we've talked to him several times, but I'm not familiar with the details of how the DevRel folded into product.

How did that actually play out? How does that roll out now? Like you said, DevRel took it over. What does that mean? I mean, so he was promoted from DevRel to product. So there is still DevRel at Vercel. It is just far, far less visible than it used to be. And probably for the better, I don't know. They basically just had attrition without replacement.

And that's just how the team sort of shifted its priorities. I mean, they needed a VP product. And Lee proved to himself and to the company that he was up to the task. And I guess they promoted him. I can't really speak for his personal experience. I only hear tangentially from him and other people, but I don't hear the full story. So you can talk to him about that.

I was less on the specifics of his specifics, but more like how they as an organization achieve that because leading product and leading DevRel is uniquely different, but also not exactly far off. Like to build the best product, you have to have a connection with the people that you're building it for, which is a function of DevRel, a connection to community.

But you also have to have business mindset, like where do we actually make money? Where do our users really get joy? Where is our business trying to go, not just where's the product trying to go? Which sometimes is similar or the same, but not always. And so I would not suspect it would be easy for a dev role just to take that over unless they've got some prior leading product, product management experience

or they're just a lead rob or they just like slay it. Yeah, again, not really speaking about his specifics, but I do think that if people care enough about developer experience, then it basically is a shadow product team anyway. This is something I've talked about again and again, which is kind of the existential problem with DevRel,

which is that you're supposed to be the voice of the user. It's supposed to be a two-way street. You spread the good word out. That's the Dev Evangelist role. And then the Dev relations role takes that good word, then you get the feedback from developers and brings it back into the company except most of the company doesn't want to hear it because

they already have backlogs and you're just adding to the backlog and you're not welcome here, go away. So a really good dev experience person would prioritize and justify and go like, you know, here's what our developers are telling us, just listen to me, I'm good at the people and I

understand man what you're shining a light on though is that friction between DevRel's job and products job yes right and so rather than fight the fight merge just take it over yeah yeah this is becoming more

this is why I asked the question because that i was less trying to understand lee rob's personal specifics but more this function of because i think you kind of clarified it there where there's that friction point if you're just kind of going out there and you've got less respect or less political power with product and direction can you even do your job well if when you go back to the table and you say, hey, I'm out there fighting the fight. I just flew 10,000 miles last month, spent, you know, three weekends on the road. And here's the wisdom. And everyone's like- Here's what the company paid for.

Right. We are paying for this. Right. And then product is like, no, we got different margin. I've been talking to users too, but in a different way. And so we're going to pause your thing because we've got enough backlog already and I've already led this direction here. So it's almost just wasted.

It's absolutely wasted. Yeah, it's absolutely. Then you go back to the developers that you spoke to. I was trying to be kind about it, I suppose, by saying almost. Yeah, and then you didn't deliver for the people you spoke to either, right? Like you couldn't actually get their request

represented in a way that gets it so you're ineffective on both sides. That can be incredibly frustrating, I'm sure. Yeah, I call this a two-way umbrella for the company to the users and from the users back to the company. And you just have to filter a lot. And so I call this an emotional burden.

When I tweeted that, I was definitely feeling it. Yeah, I mean, this comes with the territory. But if you want to actually change anything about it instead of just tolerating it, you take over product. And this is something I actually ended up doing at Temporal. I ended up being the PM of the TypeScript experience. And actually, I think it helps that sort of two-way synergy because after I was done being the PM,

then I also then flipped back to my DevRel role and started talking about the stuff that I did. So if you were heavily involved in talking to users and designing the thing, then you can very authentically say, I designed this and here's how you're supposed to use it and people believe you. And if you're that highly invested, you might as well just be repping your own product, right? That seems to be the move, right? Because now that's how you,

it's easier than convincing the product manager to do your things. It's just become the product manager and that can be very difficult unless it's your own company, in which case you wear all the hats and you bear all the burdens, but you also get all the upside.

Yeah, I mean, it's exactly what I'm doing, but I would say it's a very tough job to hold all those'm doing. But I would say like it's a very tough job to hold all those things in one go. And I think it's a very privileged position to be in to help to do that for a company that

has a lot more resources than you. So I'll just say like, yeah, people are interested in entrepreneurship. You want to be able to build and you want to be able to sell. This is one of,

this sort of dev experience, dev rel combined with product role is probably the one of the best experience, DevRel combined with product role is probably one of the best jobs out there in developer tooling. I agree. I think putting DevRel or whatever DevRel's function is, even if you don't call it DevRel under product makes a lot of sense because

I think the reason why DevRel kind of gets this shill, as you mentioned earlier, Jared, or this bad rap or this sort of pejorative feeling is that you feel like you're out there trying to sell and that's not your job I think the job of DevRel generally is try to showcase the vision of where the product is going and get that resonance from the community and see if it's landing and also create advocates out there who become passionate about where you're going so that you can become essentially take that wisdom you've got back to the team

and say, this is what we're doing. This is how people feel about it. This is where they're not getting it. This is where my demos and my tutorials are not landing. This is where my 101s are not, they're falling short. It's because of this part in the workflow or whatever it might be. Their job is not to sell their job is to tell and share the story which if you do it right does sell but you're not trying to sell even in our ad spots i don't know how much you care about these things

how we do our ad spots i literally tell these people that i sit down with more often than not ceos of the companies and i'm like i don't want you to sell. Okay. If you're, if in this conversation, you're trying to sell, we're doing it wrong. I just want you to share your story. Can you share your story for me? And not that story on my phone. Sorry about that. It is to just don't, don't sell. I don't, that's, that's my job to tell people where to go and to be excited about your thing and to give people waypoints. Don't come on here and sell. That's my job to tell people where to go and to be excited about your thing and to give people waypoints. Don't come on here and sell. Same thing for DevRel.

Don't go out there and sell. Just tell people what we're doing and get that feedback on how we're doing it. And how do we change to make it work? If I could make one tweak instead of just tell people what we're doing, you should nerd snipe them. That is the way to hook developers. Tell people

what hard problem you worked on and tell people the backstory to why you worked on it or what's the intellectual history behind these ideas. Why is this the thing that is inevitably what everyone is going to be going towards?

Whether or not it's you or you build it in-house, or you buy it from us, or someone else builds it, it doesn't matter. The industry is going this way. Are you with us, or are you part of the last century, or whatever? That is the kind of story that I like to tell, which is not just tell us what you're doing,

but put us in a broader narrative of where are we in that moment in history? And I think you get the nerd snipe. Definitely try to show a little bit of the behind the scenes. I think a lot of the people, a lot of standard marketing advice is benefits over features.

And I think there's a little bit of inversion for developers where you want to talk about features because you let the developers figure out the benefits. But go down to the implementation details because people love to learn about that so that they never have to touch it. And then go, here's the benefits of that. But if you only lead with benefits,

we will accelerate your digital transformation by 10% in the next quarter. I don't care as a developer. Show me how it works and tell me something cool. One flavor of that that I think would be interesting, which maybe we've done, maybe we haven't done, and we definitely do on some of our shows, is tell me how hard this particular thing was to build.

What did you have to go through to build this thing? Exactly. And that's where the nerd sniping comes, right? The nerd snipe is so effective for selling the product, but also selling you on working with me. Come join us, we work on Koshy. Tan, tan. Yeah, exactly.

And that's why I'm always like, I cannot tell people enough to do this because I think you have to kind of repeat it to them a lot that people want to be nerd snipes. Like they want to work on hard things. And if you if you just emphasize the nerd snipe, you accomplish both goals of doing any public appearance, which is recruiting and selling. Nerd sniping for the win. So riddle me this, how has data science not been rebranded into AI engineering or data engineering or pick your flavor of the day? It seems like the data scientists are just doing what they were doing before, mostly.

Maybe there's some deployment things going on now and just like changing the label on their business card. I think there's definitely some rebranded data scientists that are transitioning over to generative AI really well. But I think there's a qualitative difference in the kind of people that are doing really well in generative AI that have no shared history, no shared skills, no shared language

with the data scientists. There are many, many successful AI engineers that do not know Python. And for data scientists not knowing Python, not knowing Pandas, what are you even doing here? You're not part of our club. Okay, so you're saying the opportunity has broadened the industry

to where you don't need to have the same background as a traditional data scientist. And that's not to say that data scientist demand hasn't gone down at all. There's been a secular growth trend for decades. I would just say this is just a different type of skill set that makes you successful in this era rather than the previous era. And whether or not you're successful here relies a lot more on your creativity and full stack product development skills than just pure data science because data science comes in later when you have data to work with but now when you have foundation models that you can just prompt and make an mvp so quickly

you actually need to be creative and quick to market rather than being deliberative and analytical being analytical actually slows you down and makes you too conservative. You're worried, like, what are you doing worrying about costs when the cost of intelligence for GPT-3 goes down 90% a year? That kind of stuff. Makes sense.

So what are the attributes of an AI engineer? Aha. Aha. I have a convenient blog post to refer to people. Please read it out loud to us now. So yeah, obviously not to be annoying, but I do actually have a blog post for this. And that's part of the sort of meta game

I do preach to people about the learning in public, which is like anytime there's a frequently asked question, you should have a canonical blog post for it. Not just because you can be that annoying person to send it to people, but actually because you can actually spend the time to think about it. So you have a more blog post for it, not just because you can be that annoying person to send it to people, but actually because you can actually spend the time to think about it. So you have a more complete thought.

I think Kelsey Hightower often says, you don't really know what you think until you write it down. And the reason he's so thoughtful is actually he writes a lot of stuff down first. So there's the rise of the engineer post that just celebrated its one year anniversary. And, you know, it's the start of a lot of things I'm doing. And more recently, we actually published a hiring engineer post that actually published some sort of reference job descriptions for people. And it's I like the sort of framing of offensive and defensive engineering. Defensive meaning like being able to create systems that fundamentally work on top of non-deterministic AI models.

Right. systems that fundamentally work on top of non-deterministic AI models. LLMs, as you might know, hallucinate, they're non-deterministic, and they actually fail a lot. The P99s of latencies are ridiculous sometimes, just for whatever inference load reasons your selected API provider might have. And so it's effectively like, how do you create a reliable service on top of fundamentally unreliable foundations? Sounds very familiar. That's distributed systems. And like a lot of the sort of same language,

maybe slightly different tooling emerges coming out of that. That's defensive though. And then there's also, let's just call it preventing against regressions or optimizing costs. And that's a lot of fine tuning for smaller models and all that good stuff. But really offensive AI engineering

is exploring new frontiers, right? Like this capability just came out. How can we put that to good use in a sort of end-user product way that immediately clicks for them and generates a lot of revenue? I think the image companies have actually had the most success

out of this. MidJourney is my sort of favorite example of this, making something like $300 million a year with 50 employees, completely bootstrapped, like no VC funding. So for people counting at home, that's $6 million per employee. And, you know, I think there's more examples I can list in there, but it doesn't really matter. Like if the new capability comes out, is it the optimization guy or the creative technologist that wins? It's the creative technologist. And for me, it's like, okay, most engineers are not creative technologists,

but are they product people? Can they think about, how do I use this capability that just emerged to solve problems for a customer in some way? They can be more creative there. So I'm trying to basically explain why that is qualitatively different than the data scientist role, which is mostly analytical, which is still very important. It's just like a different skill set that like, if you just don't

have that gene in you of like being creative as a product thinker, then you won't be as successful as someone who is. Who are some other people besides MidJourney who are very successful? Because from my perspective, I've seen, set ChatGPT and the alikes aside, generally use chatbots as a category, set that category aside. Obviously, huge success, right? Lots of value, etc. Give you that. MidJourney, give you that one.

But the companies that have brand new products that are making moves in the marketplace that have gone beyond demo and hype to actual product people are paying for, I don't have my thumb on that pulse. I'm not seeing much of that. I'm sure you're seeing more of it. So that's what I'm asking about specifically.

Yeah, so my bar for, let's have a bar, right? For like, it's easy to get to production on a small use case that nobody cares about. So production to me is not good enough. So let's have an even more aggressive bar of it must make $100 million a year. Like that's at a point where you can IPO as an independent company, right?

You know, maybe the bar is 200, but you know, that's just a factor of two. So let's just say $100 million. What AI use cases have made $100 million a year? So obviously we talked about mid-journey. I have four, and then the fifth one's more speculative. Conveniently, this is another blog post called

The Anatomy of Autonomy of People Looking This Up. But generative text for writing, JasperAI and Writer.com, both have about 100... Jasper reached the 75 million ARR before they imploded, and Rider.com I think is comfortably at 100. How do they implode?

Adam, did you miss this? What happened with Jasper? I don't know what their revenues are today, but effectively they got rug pulled. They got acquired. Well, they imploded before they got acquired. The acquisition was the exit. Obviously, I'm just saying secondhand stories from other people.

So don't quote me on any of this. But effectively, the founder... We're on a podcast being transcribed. Yeah, but I'm transcripted. Just said, don't quote me on this. Don't quote my transcript. The founder sold a whole bunch of secondary and it just peaced out

so he basically lost interest in developing the company but then also it seems like they so they built a very successful business on top of GPT-3 before ChatGPT and then a whole bunch of people found out after ChatGPT that they weren't actually doing that much

on top of GPT-3 and then they migrated to ChatGPT so they were basically killed by ChatGPT that they weren't actually doing that much on top of GPT-3 and then they migrated to ChatGPT. So they were basically killed by ChatGPT is the common narrative. I don't know how true that is because their focus was very, very strong on e-commerce on Facebook. The reason you don't hear about them because you're not on Facebook. They are. They did very, very well. They went from zero to 75 million revenue in two years. Very few people have done that. But anyway, so since then, the emergent winner in that sort of general text for writing category is writer.com. They seem to have figured out that sort of post-chat GPT navigation, which is not hard. Focusing on users and building differentiated features on top of the model is the job of AI engineering. And you just have to do a more creative, more dedicated job staying on top of the model is the job of AI engineering. And you just have to do a more creative, more dedicated job staying on top of it

and not being defeated by OpenAI's first move into chatbots. So I don't know if that's a fair... I really want to stress, I don't know. This is not my industry. I don't know this specific writing case, whether that's a fair characterization of what Jasper went through.

But it is an interesting story. But like, so fair amount of revenue there. Copilot, now I think 200 million in ARR. So like,

well past, right? And there's a bunch of other smaller Copilot competitors all with like decent revenue, many of which spoke at the

engineer conference that I held. So you can go look at that. ChatGBT, I think something like 2 billion a year in revenue. I ruled that one out. So you can go look at that. ChatGPT, I think something like $2 billion a year in revenue. I ruled that one out.

So those are the four categories that we are very, very sure make sense. There's a bunch of co-pilot for other knowledge worker type things. Harvey is now the emergent example for we are co-pilot for law and every lawyer needs this or you are behind. Fine. So for every knowledge work profession, there will be a co-pilot for law and every lawyer needs this or you are behind. Like, fine, right?

So for every knowledge work profession, there will be a co-pilot for X, right? And each of those things will easily make $100, $200 million because you are replacing a whole bunch of junior workers for that. We can talk about the replacement theory issue, but like there is real revenue here. There's a real case for generative AI.

It does not have to get smarter to be useful. Okay. The fifth category beyond all this is the agents category, which is the most contentious one, right? It was a complete bubble last year. This year, the bubble company is Cognition, Devin, also spoke at my conference the first time they ever spoke at a conference. I like them. I actually have access and I use them. We can talk about Cognition if you want. They're not the only players in this game of like the fully autonomous agents. This one happens to be code related, but there are others that are not code related. I do think that whoever eventually cracks this will be able to make significant revenue, but we haven't seen it yet, obviously.

But the bar is, for everyone listening, is can it make a hundred million dollars? And if that's not good enough for you, nothing is good enough for you, right? Like, you know, if your bar is higher than mine, then you're just going to have to wait longer to see the results. But this is happening in progress and, you know, you can either criticize it from afar or you can just get in earlier and track the progress as I'm here with Brandon Fu, co-founder and CEO of Paragon. Paragon lets B2B SaaS companies ship native integrations to production in days with more than 130 pre-built connectors or configure your own custom integrations. Brandon, there's a certain level of pain that a product team or an engineering team has to endure to, let's just call it rolling your own

integrations. Help me understand that pain, that angst for those teams. Help me understand that true pain of delayed integrations for a product, not integrating or having to roll your own integration, this seemingly slower route to integrations. I think for context, one of the reasons we started Paragon is that today the average company uses over 130 different software applications. So that means if you're a B2B software company selling into the market, there's over 130 of your customers' applications that you probably need to connect your tool to. Because customers today expect that any product they buy is going to work seamlessly with the hundreds of other applications that they're using. Of course, we see this when companies come to us and they say, hey, we have a backlog

of 10 or 20 or 50 integrations that, you know, our sales team has told us we're losing deals because customers are asking us to integrate with all these different apps and we can't deliver on those integrations or maybe our competitors are integrating with these tools. And the problem that that results in for product and engineering teams, of course, is how do we build and maintain these integrations in a way that's scalable, that we can not just satisfy what customers are asking for us today, but we can maintain those integrations in a way that's scalable, that we can not just satisfy what customers are asking for us today, but we can maintain those integrations in a way that's scalable for, you know, the next hundred customers, the next hundred integrations that we need to build. So for engineering, one of the challenges, obviously the backlog and prioritizing time

for certain features or integrations, but then there's this other side where you got to really learn every single API and everything is hand rolled, custom maintained. And over time, that kind of gets I got to imagine kind of taxing on teams. What do you think? So most engineers know that, you know, every API is completely different, can be completely different in terms of how they handle authentication, in terms of how they deal with different record types. And so it becomes this problem for engineering teams to basically have to become experts in other people's APIs and what could be dozens or hundreds of different APIs. And to build those integrations we've seen can take as much as three to six months per integration for a developer to write the code to build that integration. And it depends on the use

case, of course, and the type of product that you're integrating with. But of course, that becomes a massive challenge at scale when you're looking at how do we scale our product to support 10 or 20 or 50 different integrations. So again, Paragon was really designed to solve that problem and to distill the complexities and the nuances and the differences between hundreds of different SaaS apps into a single connecting platform, into a single SDK that your engineers can install in your app and then easily connect your products to all these different SaaS applications in the market. Okay. Paragon is built for product management. It's built for engineering. It's built for

everybody. Ship hundreds of native integrations into your SaaS application in days. Or build your own custom connector with any API. Learn more at useparagon.com slash changelog. Again, useparagon.com slash changelog. That's U-S-E-P-A-R-A-G-O-N.com slash changelog. I would love to hear more about Cognition and Devin. It seems like they were unscrupulous in their marketing

with the Upwork thing. Oh, God. Okay. I will defend them here. So, yes, the headline on Hacker News reads, cognition or Devin debunked, right? Very nice alliteration there. Out of the nine videos that they produced, one of them was overstated claim, which I agree they should not have put out. And the claim was that this bot could make money on Upwork autonomously.

Yeah. Paste in an Upwork job and it would just do the rest and make money for you. That was, obviously there was a human behind that being the bridge from Upwork to the bot and also the bridge from the bot to Slack which it does not, Devin does not have Slack integration. Some stuff in the video was not

the true Devin experience or they failed to show. It's how when people market games, they tell you if it's to show. You know, it's how like when people market games, they tell you if it's like in-game render or if it's just, you know, some artist rendition

of what the game should feel like. And that one was definitely the artist rendition of what the game should feel like eventually. But yeah, I mean, like one video was inevitably produced.

The guy who made it owned up to it and said like, yeah, sorry, I should not have done that. But that doesn't take away that this is still the most significant agent we've ever seen outside of OpenAI. Prior to this, my reference for

best agents outside of the self-driving cars that we have in San Francisco, because those things are the best agents in the world. Second best agent in the world was ChachiPT Code Interpreter. Since then, we have Devin, and since then, we have Cloudvin, and since then we have Cloud Artifact from Anthropic, which you can talk about. But Devin is really good.

It's a really, really good agent. Actually, a really good generalist agent, not even factoring in the code writing ability. And I hope that people don't throw out the baby with the bathwater because unless you've actually tried it, you don't know what you're talking about. You're just reading headlines and you're just repeating the last headline that you just read. Well, we can't know what you're talking about. You're just reading headlines and you're just repeating the last headline

that you just read. Well, we can't try it because we sign up for a wait list and then they don't give you access to it. And so what do you want us to do besides speculate? We can't. Maybe spend less time on things where like, you know, you're just kind of repeating headlines.

I'm not spending any time on it. I watched the Upwork video. I watched the debunking video. He certainly debunked what they did and there was no question to it. So I understand that you're okay with nine out of ten times i tell the truth but when i'm coming out and try to make a splash and i'm lying in my marketing material sorry i'm just gonna go ahead and just remember i'm just gonna go ahead and remember that that was a bad idea and they should have done it yeah still like it's a good product

you know i have to square those which i can, I have to take your word for it. Yeah. I have the fortunate ability to say I have no vested interest in Devin. You know, I just like, they gave me access, I used it,

I was impressed. And so was Patrick Collison, so was Fred Urson from Coinbase. There's like a bunch of people who cannot be bought who like it. You know.

Sure. That's good. That's good. I'll take your word for it. I'll take your word for it. I can't do anything else. So can we, but can we talk about like. I'll take your word for it. I'll take your word for it. I can't do anything else. So can we, but can we talk about like,

like I think the technical design of it can be replicated. I think the real question, the thing that people really should be talking about instead of the video, which was a mistake, one-off mistake,

the most structural issue with Devin is can it be cloned? How thick is their moat? This is a six month old company that is now valued at $2 billion, right? Which is absurd by any stretch of the imagination. So that's the real question which Devin has to answer and the rest of the AI engineering industry has to answer.

There's a project called OpenDevin that is trying their very hardest to replicate it. I've interviewed both of them. You can check it on my podcast. I would say that Devin is still ahead. Who knows how long it's going to last. But I think that the sort of structural merits of what Devin has innovated

in terms of how agents should be interacting with each other, what are the necessary components of agents, that is going to stick. And if you focus too much on the marketing video, you're going to miss the actual lesson to be learned from Devin, which is that, hey,

your agent should have a coding environment, should have access to a browser, should have a plan, and should have, let's say, like a terminal and chat, interactive chat bot where you can sort of observe what it's doing and correct it in real time and it can respond to you in real time. That is the UX that has wowed all these people, wowed myself. I have never seen it in any other agent before. And I think it's going to be the standard or state of the art for all agents going forward because it's so good. What are the odds that something like that, which is very general, as you said, just gets Sherlocked by OpenAI?

In a way, it has been, but not by OpenAI, it's by Anthropic, right? Which is the other thing that I mentioned. So that's Cloud Artifacts is the other thing that people should really think about. They definitely looked at Devin and were like, oh yeah, we're taking a bunch of that. They did not do the browser access because these guys are way too worried about safety as compared to me and as compared to Scott from Devin. So Cloud Projects is basically an advanced version of ChatGPT's Code Interpreter that can render a working web app. I often say the sort of spicy version of this is that Cloud Anthropic did more for

code agents in two months than Replit has done in two years because it's basically Replit. So for the record, for my Replit friends, obviously they did not build a full sort of REPL environment in IDE. Anyway, still, you can do very

significant programs in Cloud now that you could not do in ChatGPT Code Interpreter. You can do in Devin, but Devin is slower than Cloud and less generally capable than Cloud. It's just very, very good. And for the first time, people are actually openly talking about

Anthropic being better than OpenAI. OpenAI has lost its crown as the undisputed number one, which is wild. Like I did not expect a year ago to be living in this world, but now we do live in a world where it's like a multipolar world

where there are multiple sort of top powers in this space. It's very, very good. And you can try it. Unlike Devin. Yeah. No, that sounds good. That does sound good.

Love some competition for OpenAI of course there's been you know turmoil over there as well and there's been interesting things going on inside and around

OpenAI maybe for the engineers and you know listening I would say like the progress here has been at the model layer

you know so Cloud Artifacts is built on top of 3.5 Sonnet which is the current world best model but also

there's a significant amount of AI engineering that was required to build Devin and to build artifacts. And I think that if you want to see what the future of AI engineering should look like, you should be trying to build a clone of this thing. That's what I'm trying to do. Because I think a lot of AI engineering will look like this. Will look like, how do you wire up a model to the real world to produce projects of significant value that you would otherwise have had to assign

to a junior engineer? I think that is absolutely the sort of gold trophy that people are going for right now. And obviously, the step beyond that is artificial general intelligence, but this is a pretty good second place. A $2 billion market cap in six months is absolutely amazing.

It's a bubble. But still, the fact that it took six months tells me it's going to take less time when more people are applying to that. Is there actually a moat there? Time will tell, I guess. Time will tell. They're trying to build one. This is a question of business and less about tech. time will tell, I guess. Yeah, time will tell. They're trying to build one. I think the, yeah,

so this is a question of business and less about tech. And the moat is really user data, right? The more people you can get coding with this thing, and the more you can observe how people interact with these agents, like Devin has a six month head start on everyone else

on how people work with Devin-like agents. And if Devin-like agents agents is the goal of this thing, then they will have the best RLHF feedback data on the planet for specifically this task. It's the same motivation that OpenAI had with ChatGPT, which is, they kind of lucked into this, but now they have the longest series of chat oriented data sources, human feedback data that you previously had to pay a lot for. And then, so that mode is the data mode, but then actually it also becomes you're investing ahead of where the capabilities are.

So you're sort of saying like, I will build code, I'll write manual code to build out the capabilities that I don't have yet. But as the capabilities grow in the fundamental model, you can just kind of swap them out for your sort of handwritten code and be more generally capable with the scaffold that you already built ahead of time. So I feel like I'm being vague there, but you'll see this in the form of the model's ability

to interact with the real world. A lot of times you're writing integrations, you're writing it'll interface with OpenAPI. Screw that, man. The future is just models surfing websites just like anyone else would surf websites and interacting with them exactly like you would interact with them. But right now

we have to use the crush of code. In the future, we don't. Models surfing websites. How does that sound, Adam? Dangerous, cool, amazing, awesome. It's a new world. On the grander timescale of this, this is happening within the last three years.

What does 30 years of this do? What does 300 years of this do? We are birthing a new life form. I do think about that timescale as well. It's just an exciting time to be alive and to observe this. I don't think it's useful to try to resist it because it's happening anyway. I think this is why alignment is important because the people who have believed in this way earlier than anyone else is the alignment people. They took AI safety more seriously because they knew this was coming and the rest of us are just waking up now. And the current mindset is how do you control something that's smarter than you?

Because it's going to be. And so I think that is probably the right mode to think about it. The relevant paper for people interested is the weak to strong generalization paper from OpenAI, which is written by Jan Leike before he left for Anthropic. I do think if you're worried about the safety elements, people are working on it. They are trying to look for similar minded people. can go apply for those jobs well there's always the plug right the plug just pull the plug yeah like the legit electronic plug unless we give them the the new life form we're birthing as you've just eloquently said uh which i'm taking it back

by but i also want to dig into because it's like, wow, are we really creating a new life form? Kind of. Like, what is it that would give it autonomy, the AGI-ness of it, I suppose? And this is the holy grail question everybody's doing. We all need to partake in some marijuana before having that conversation. I know. So a lot of AI discussions tend to devolve into existential risks and AGI discussions. And part of my goal of defining AI engineer is to create a space where those discussions

are the side discussions and not the main thing. Because we're all here to engineer. We're all here to build for today's problems with today's capabilities. And I think that's a lot of how I think about my impact in this field, which is how do I guide people in a more positive direction that basically nobody's against. A lot of DevRel is the features here is not evenly distributed, but a lot of engineering here, especially in AI, is the feature is here, but it's not evenly distributed. And how do we distribute it best to everyone else? I come from Singapore. One of my favorite stories to tell is that the

Singapore government is embracing AI really well for their older folks, the people who don't speak English, the people who are disabled, the people who need the natural language interfaces to the many, many digital forms that are coming up in our lives. civil service, I think is the best form of how we do AI engineering. We don't have to go to the freshman dorm room conversation of are we bootstrapping a life form? That's fun to discuss. Happy to engage with that, but why I try to keep it to the engineering conversation is to let people have a way to ground their conversation in what can we do today.

But you're the one who said we're literally creating a new life form. Yes. I do also believe that. So you opened the topic. I'm sorry. That's okay. Which I think is, well, and I'm, so I've been silent for quite a bit

because I'm listening quite well and I'm just slurping up all the things you're saying. And then I'm also feverishly trying to find where Mark Racinovich said in our conversation with him, Mark Racinovich is the CTO, I believe, of Azure, right, Jared? Correct. So we met Mark at Microsoft Build 2024, where it was just like basically all AI everywhere, all in on AI, as we said. And I'm thinking like, well, Mark is part of Microsoft and they're one of the largest companies that benefit well from OpenAI's innovations. Sure, the discussions around cognition as well, Devin. But Mark said, I am not, I'm paraphrasing because I couldn't find the quote and I was hoping I could find it.

But Jared, please fill in the blanks. Mark said, paraphrasing again, that he is not worried about ai taking over developers jobs but then you just say we're literally birthing a new life form and then you're speculating slash revealing to some degree the the agent os of cognition and devon and what you think would be a good outline for anyone trying to copy what they've done. Meanwhile, saying they have, you know, a leg up in terms of time frame, six months. A lot of time in today's world, but realistically not a lot of time. And then you throw out, what was it?

Two billion? Three billion? What was the valuation? Two. Two. Which is just incredible. One, what is that number based on? Is it based on somebody who is willing to purchase it for?

Is that the valuation? Is it based on Tain? Yeah, Funders Fund invested at $2 billion. I think they gave them a few hundred million or something. Gotcha. Okay, so the valuation is based on venture capitalists coming in and saying, okay, we'll give you X at X valuation. I guess I'm just camping out there and sort of sitting back thinking like, gosh, is this really a new life form we're birthing? And if so, I think we got to talk about that. square the circle here because Swix was talking on a very long time span and I found the exact Mark Russinovich quote. And he said, I can tell you we're not at risk anytime soon

of losing our jobs. So maybe that harmonizes your stance Swix, what do you think? Yeah, absolutely. We should always be clear about what time span we're talking in and there's a big difference between the near term and long term. I just think that in the grand scheme of things, if most AGI timelines

by the way are like by 2050 we shall have AGI. That's within our lifetimes, guys. It's time to panic if you really think this is going to end humanity. It's time to panic. Seriously, we have Eliezer Yudkowsky saying ethically the right thing to do right now is to bomb all data centers in the world because humanity ends otherwise. He said this in a New York. I mean, I guess if we're in charge of this in terms of innovating it and creating it, how can we not have fail-safes in place to be in charge of if it goes wrong?

I mean, I think that's where it has to come down to because I jokingly said pull the plug, but I literally mean if we control the physical hardwired plug into the wall. Now, Jared, if that book I mentioned in the intro that I shared with you a while back, which I can happily share here too, if that happens, then we're in a different world. I speculated a good intro to a book or a movie and I'm thinking more movie than book but all good movies tend to begin as books. Sometimes they get, they're bad movies of good books. But anyways, I digress. I was speculating that this intro scene to this movie

was a very beautiful cinematic scene where you see this human being and it's so strange to say things like this a human being is happily racking and stacking these servers happily organizing this hardware happily instantiating a new machine into the rack meanwhile the entire task was given to the human by artificial intelligence so the boss you said before live above or below the api i think we're kind of like nice callback that's a good callback i forgot about that a version of that is like above or below the ai you just take out one letter oh because at that point it's like well in the future this dystopian potentially non-dystopian

future we're subjects of ai but only if we allow it but if we're in control of the hardware and we're the physical beings for now because you do have uh boston dynamics out there creating robot dogs and the latest version of atlas like what point do we lose, I guess? That's why I'm saying bomb the data centers, man. Bomb them. Pretty much. The question is, can we pull the plug

on these bots? For what it's worth, this is my favorite joke in this category, which is Sam Altman is very well known for carrying around a blue bag. Everyone's joke is that the button is in there. If he never needed to push the button, it's in the blue bag. I don't think we can because the secret's out that it's mostly possible to simulate intelligence inside of neural networks. And even if the current transformer paradigm

doesn't really pan out for that, something else will. Because we evolved from non-sentient life forms, we think. Unless we were created from, in the span of seven days. So if we can evolve, something else can evolve too.

And we are currently speed running evolution of this particular life form. So I don't think that's necessarily a negative for us, except that in every prior incidence of a more primitive civilization encountering a more advanced civilization, the more advanced civilization accidentally wipes out

the more primitive civilization. And right now, AI is not more advanced than us, but it is growing much, much faster than us. It is spreading much, much faster. It learns much faster than us. And so we need to figure out how to contain this

or eject it from our solar system so it doesn't affect us. I don't think these are, I don't think that's possible so we have to contain it. We have to align it.

That's the only way. So yeah, and also, I also don't think that capitalism and this safety is this sort of top-down safety are aligned in a sense that in order to control this, if you really are concerned about safety, you have to nationalize all AI labs. And then you cannot stop there because what uses nationalizing things within one border, you have to nationalize all borders.

So you have to take over the world and control all AI developments if your intention is to really control from a top-down basis of all AI safety. So that's not happening. Yeah, borders are a big concern. This is the classic, like, China's going to do it if we don't do it. Well, there's this show out there called Westworld. Have you seen this show, Westworld, Swix? Yeah, great season one and two.

Okay, you've got to watch season three. Isn't there a season four? There's going to be a season four. There is? No, actually I think there was going to be a season four, but I believe it was canceled. I don't know. It's an HBO show. I got to check in on that. So in my opinion, the entire show is worth watching for season three alone. And I think you only really need to maybe even watch recaps of season one

and season two to watch season three i don't think you're really it's almost standalone in my opinion and i think anybody out there listening to this right now head nod into season three is is nowhere i'm going and i don't want to ruin any of the plot for you all because you haven't watched it but i would say go watch it a lot of what you're talking about here is represented some way shape or form in the intelligence and the autonomous beings let's just say that are out there in the world doing different things and it's very captivating from a cinematic standpoint and i think if we're 26 years away from 2050 i had to do the math there real quick if we're 26 years away from agi or had to do the math there real quick. If we're 26 years away from AGI or even the beginnings

of it, and cognition can create what they created in six months or some span of time less than a year, I gotta imagine whatever was in Westworld Season 3 is closer than we think. Some version of that is closer than we think. It could be 2070.

Yeah, plus minus 20 years i don't even know how to do math these days like yeah i mean like 30 more years after that 20 more years after that it's got to be close if you get to that speed of creation and then i would also say the other thing i've learned about is von neumann probes it's this idea of a self-replicating spacecraft so shooting it out into space is not going to be helpful because they they might they might allow themselves to escape on a von nomen probe which will just self-replicate it will begin to ore and mine planets to create new materials to create themselves to just replicate and come back and do whatever. Now they could be

peaceful if you've read the books I've read. No real response to any of that apart from it'll happen. Really, if you want to be a player on this stage, you either need to be a political leader of a world power

or you need to be a head of a major AI research lab. Basically, the rest of us don't really get a say. This is not something where democracy has any sway over. We are below the ABI. Below the AI line. I think we're above it in the sense of we do get freedom from, currently we do get freedom from the mundane tasks. I no longer care about doing really minor features because I can just tell Cursor to do it for me and it does it really well. If Cognition pans out or something like

Cognition pans out, then I will have a lot of PRs done purely by agents and that's great. We will live below, we always have and always will live below the power line. That's separate from the API line of people actually deciding the

sort of future course of humanity. And like, I think where engineers really make or break here is whether or not we choose to join them and enable them because they still need us to execute things. The one hope, the one note of optimism, you know, between the very short-term future, which is where we are today and a very long-term future, which is when AGI is here, is that I do think that AI engineers are the last job to exist because they are the job, mathematically, to eliminate the other jobs.

You need AI engineers to eliminate the lawyer. You need AI engineers to eliminate the, I don't know, the executive assistant. If you're worried about job replacement, go be an AI engineer because that will be the last job. Then we'll be post-abundance and then we can explore the stars, but until then

you should be an AI engineer. There's the sales pitch. If you would like to destroy all other jobs, become an AI engineer and you will be the last person standing. I do have to bring out my favorite TV show, Jared. Silicon Valley. Well, you lost me at Westworld. I do have to bring out my favorite TV show, Jared.

Silicon Valley. Well, you lost me at Westworld. I haven't seen a single episode. Don't know what you're referring to in season three. So go ahead, man. In the final episode of season seven, sorry, season six, episode seven, called Exit Event of Silicon Valley.

They're locked in the Pied Piper offices and they're dealing with what they're dealing with. Obviously it has to deal with AI because that's the conversation we're hearing right now. And Jared says, okay, is this a good thing or a bad thing? Somebody tell me how to feel. And Guilfoyle says, and this is my favorite line ever, abject terror for you. Build from there. So that's my advice for everyone that is not a political power or whatever you just said you had to be, Swix, to have any say in the future of this because abject terror for you, build from there. Yeah. I don't call it terror so much as we live in a point of history, right? History is happening. We're lucky to be alive to witness this in this moment. We have some minor sway on it,

but history is bigger than us and it's going to happen. It's going to take course. And I don't know. To me, I think this is part of the general EAC message, which is that if you're pro-life, you don't have to be only pro-human life. You're pro-life in any life form. You're pro-consciousness in any form. And if humanity happens to be the sort of bootstrap load sequence to what actually is what life is supposed to be, which is sort of more reliable, sustainable, faster learning machines than us, then maybe that's the natural order of things. I don't know. I would like it to not be the case because I like humanity. I like my body. But we do live in a world where that is a possibility. And this is why we almost outlaw conversations on artificial intelligence on this podcast. Because of this. This is almost why we outlaw it.

Almost. I'll mention one last thing, maybe as a positive parting thought. Please be positive. I'll help you be positive. We used to basically completely throw in the towel on interpreting the model weights. You know, GPT-3 was 175 billion parameters. Absolutely, like just meaningless numbers. Like 175 billion meaningless numbers.

And I used to just think of mechanistic interpretability as a joke. I will say Anthropic has done a crazy amount of work here recently to make features of models interpretable. And if we can study the brain of these things as they think, then we can control them very, very effectively. And I have gone from this will never happen to, oh, I didn't know that this is possible. And that's where you should read the paper Scaling Monosemanticity from Anthropic, which they demonstrated they can do it on CloudSaudit, which is a mid-sized model. We think

it's something between 15 and 70 billion parameters. If we can do that to 15 to 70 billion parameters from a standing starting point of less than 100 million parameters last year. We are accelerating our ability to interpret models faster than our ability to grow these models. And that is a good thing. We will fully understand and map this brain before it is bigger than us. And so we will be able to control it if that is true. The trajectory of interpretability this year has been an unmitigated success story, and it is going to get better. And we might actually overtake our ability

to grow these brains. And that will help us control these programs much more effectively than basically any other method possible. I did not know that. That's very cool. What was the name of that paper again? Scaling Monosymmetricity. It's the third in a trilogy of semanticity papers.

The first one is Superposition. I covered that in my AI News newsletter. This is where I plug my newsletter for go subscribe if you want to keep up on this stuff because that's my daily pick of what the top thing to know is. All right, Swix. Well, fun times, great conversation.

DevRel, AI Engineering, AGI, The end of the world. All the things. We expect nothing less. Hook us up with links to your newsletter, to your pod, to all the things mentioned, and we'll make sure they hit the show notes for folks to follow up and connect with you on the interwebs.

Yeah, thanks for having me on. I like this Enfrance format, because then we can just talk about whatever is top of mind instead of sticking to a specific company or piece. All right. Well, that's all for now. But we'll talk to you all on the next one. Bye, friends.

OK. AGI is coming. 2050. Mark it on your calendar. I know I have because I'm kind of scared. As Guilfoyle said, abject terror for you. Build from there. I don't disagree.

Honestly, I'm a little bit terrified of, like, what will happen when the reality of an AGI comes. Especially when you contrast that against Boston Dynamics' latest HD Atlas update. I mean, it's kind of terrifying to think about that kind of robot potentially having AGI built in. Okay, enough doom and gloom. DevRel is here to stay. We've defined it. We've described it. But Zerp is real and Zerp affects everything. And by the way, there's a bonus on this episode for our plus plus subscribers. Learn more at changelog.com slash plus plus. It's better. It is better. Again, learn more and join the fold at changelog.com slash plus plus. Okay, big thank you to our friends and sponsors at Sentry, at 1Password, and also our friends at Paragon.

And a massive thank you to our friends at Fly.io. Over 3 million apps are now launched on Fly. As we said before, we're one of them. Learn more at fly.io. And thank you to BMC, Breakmaster Cylinder for those awesome beats. Well, that's it. This show's done.

We'll see you next week. Yeah, so like basically started a podcast February of last year. We had a very good initial success right up to when we had George Hotz on. We actually hit number 10 on the US tech charts. So we actually overtook the A16Z podcast. Since then, we haven't really had a ton of hits, just slow, steady wins, I would say. And I feel like it's a little bit of a rut. I think Latent Space has done well serving its niche of, we are sort of the AI engineering

podcast. Tobias Macy from the Data Engineering podcast and Python podcast tried to get into the AI engineering space and ended up giving up on it. So I think we have stood the test of time where maybe it's mildly differentiated from practical AI, which is the show that you guys run and I'm wearing the shirt off today. I do think there's the metagame of, all right, how do I run my existing operations better?

Which is I need better editing. I need to do better on YouTube. I need to figure out my audio quality a lot better. And we can talk about all that. Then there's also like, that's the defensive side. Then the offensive side is how do I grow? Like how do I get to the next level? And for the growth element, it's very obvious.

Book big names. That's it. Like that's basically it. It's the only thing people care about is just hearing from big names and I can go get them. But like like i just feel like then i'll just kind of devolve to like an interview show just like everyone else i'll just be one of many interview shows so yeah that's that's where i'm at that's where you're at well that's further than most people get so uh congrats on the success