The AI Daily Brief: Artificial Intelligence News and Analysis - AI Operations and AI Engineer: The New Careers AI Is Creating

Episode Date: October 9, 2023

On today's episode, NLW explores the ways AI is changing people's career paths. In particular, he examines two entirely new roles -- the AI Engineer and AI Operations -- and what they mean for you. Links: https://www.semafor.com/article/10/04/2023/ai-is-spurring-the-rise-of-the-novice-coder https://www.latent.space/p/ai-engineer https://twitter.com/karpathy/status/1674873002314563584 https://news.theaiexchange.com/p/premium-defining-ai-operations-new-cohorts-announced https://twitter.com/rachel_l_woods/status/1684203209475121155/photo/1  TAKE OUR SURVEY ON EDUCATIONAL AND LEARNING RESOURCE CONTENT: https://bit.ly/aibreakdownsurvey ABOUT THE AI BREAKDOWN The AI Breakdown helps you understand the most important news and discussions in AI.  Subscribe to The AI Breakdown newsletter: https://theaibreakdown.beehiiv.com/subscribe Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@TheAIBreakdown Join the community: bit.ly/aibreakdown Learn more: http://breakdown.network/

Today on the AI breakdown, we're exploring the new careers that generative AI is creating. The AI breakdown is a daily podcast and video about the most important news and discussions in AI. Go to Breakdown.com. Network for more information about our YouTube channel, our Discord, and our newsletter. Hello, friends, quick note before we dive into the episode, today is sort of nominally a holiday in the U.S., at least it's a bank holiday. And so for this episode, I'm not doing my normal brief followed by the main episode. We are just doing a longer main episode about what I think is a really interesting topic. We will be back to our normal schedule and format tomorrow. Enjoy.

Welcome back to the AI breakdown. One of the big questions around artificial intelligence is how it's going to impact people's careers. In fact, part of what has made this such a radical growth industry with so much attention focused on it is that unlike many hypey technologies, the first time, the very first time, in fact, that many people use a tool like chat GPT or a mid-journey, they have a sense that the way that they're going to do their jobs is very, very likely to change. And some, in fact, probably expand from there and think to themselves, well, maybe the entire nature of the jobs that we will be doing as a whole is likely to change. A big goal for me with this content is to help people

navigate that transition, whatever it ends up looking like. And so today we're going to explore how AI is already transforming how people think about their careers. And in particular, what new types of roles it's actually creating. Now, part of the inspiration for this piece was a a recent article in Semaphore called The Rise of the Novice Coder. Can AI turn every employee into a developer? It sets the scene with the story of Leo Segarra. Gara was a 36-year-old father of two who was previously the building maintenance supervisor at Vanderbilt, but who, thanks to AI, has now built more than 20 software programs that Vanderbilt is using for things like faculty onboarding, tracking transactions, and the way that he did this was with an AI-assisted

program. He used the power platform for Microsoft, which is a no-code platform that is designed to help people who aren't coders actually build software apps. Now, apparently it was Microsoft's event for this piece of software last week. And so Semaphore also covered some of the other ways that people are using it. From the piece, Charles Lamana, who runs business applications for Microsoft, said some of the biggest users of power apps are in corporate finance or human resources department. He said Toyota has become a major Power Apps customer because it's instructed its employees to use the service to make improvements for logistics or other areas. Lamana said power apps are filling in the gaps not met by companies making business software.

He said digital demand is basically unquenchable. The more SaaS services you have, the more you have gaps between them. Now, where the author read expands this article out to is a discussion of just how much coding is perhaps the core use case or killer app of the current generation of LLMs. He writes, coding, that unexpected LLM skill may end up having a much bigger impact than the natural language component that has captured most of the public attention. It isn't just Microsoft's GitHub pioneering the space. Companies like Replit and Sourcegraph, among others, are getting traction by focusing on generative

AI software development. Other companies like Nomad Data are using code to supercharge LLM prompts. For instance, you might ask a chatbot to gather a bunch of data from a whole slew of corporate documents. Behind the scenes, the LLM has been instructed to create a Python script to gather the data. The combination of automated coding with natural language is a potent mix that hasn't really been exploited yet. And then here is the key line from Reed.

This likely won't lead to software engineers being replaced by AI, at least based on what we know about the technology today. More likely is the incorporation of code into more aspects of daily life and more people participating in that transition. It's taking full advantage of the powerful computers around us, both at work and at home. For years now, companies have been yearning for employees to become citizen developers and take it upon themselves to build software and improve productivity. Automated code generation could help make that a reality. We often call today's tech consumers users. We may have to come up with a new term for what comes next in the age of AI. directors might be a better one.

Okay, so in our exploration of how AI is likely to impact people's careers, one question that this piece rises up is to what extent average professional non-coders right now will have development become a part of their skill set, presumably in an AI-assisted way. Well, rather than fully trying to answer that question, let's actually continue on and talk about how AI is impacting people who are already engineers. As a way to frame that, let's read a piece by latent spaces SWIX back from June called the rise of the AI engineer.

Swix writes, Emergent capabilities are creating an emerging title. To wield them, we'll have to go beyond prompt engineering and write software. Swix begins the piece, We are observing a once-in-a-generation shift right of applied AI, fueled by the emergent capabilities in open-source-a-appi availability of foundation models. A wide range of AI tasks that used to take five years in a research team to accomplish in 2013, now just require API docs in a spare afternoon in 2023.

However, he continues, are no end of challenges in successfully evaluating applying and productizing AI. He points out evaluating and selecting which models, be it GPT4, Claude, Hugging Face Lama, or something else, tools, from the most popular chaining retrieval and vector search tools like Langchain, Lama Index, and Pine Cone, to the emerging field of autonomous agents like AutoGPT and Baby AGI, and then of course there's the news. On top of this, he writes, the sheer volume of papers and models and techniques published each day is exponentially increasing with interest in funding, so much so that keeping on top of it is

almost a full-time job. And this is where Swix gets to the culmination. I take this seriously and literally, I think it is a full-time job. I think software engineering will spawn a new sub-discipline, specializing in applications of AI and wielding the emerging stack effectively, just as site reliability engineer, DevOps engineer, data engineer, and analytics engineer emerged. The emerging and least cringe version of this role seems to be AI engineer. He continues, every startup I know of has some kind of discuss AI Slack channel. The thousands of software engineers working on productizing AI APIs and OSS models, whether on company time or on nights and weekends in corporate slacks or in indie discords, will professionalize and converge on a title, the AI engineer. This will likely

be the highest demand engineering job of the decade. AI engineers can be found everywhere from the largest companies like Microsoft and Google to leading edge startups like Figma and Notion, to independent hackers. They are making 300K doing prompt engineering at Anthropic and 900K building software at OpenAI. They're spending free weekends, hacking on ideas at AGI House and sharing tips on R slash local Lama. What is common among them all is they are taking AI advancements and shaping them into real products used by millions, virtually overnight. Not a single PhD in sight. When it comes to shipping AI products, you want engineers, not researchers. In the next section, SWIX describes how an AI engineer is different

from a machine learning specialist or a data engineer. And basically, it comes down to this idea of applying or productizing AI, as opposed to building out the underlying models. Now, what One interesting note is that this isn't the much-hyped prompt engineering job that so many have discussed. In fact, we'll talk a little bit more about that in our next section. Instead, this is something that is more technical. It really is a subcategory of engineer, not just a non-engineer using AI tools to now be able to code. It's a very different role, in other words, than what we were talking about just a moment ago. OpenAI's Andre Carpathy actually commented on this post.

He wrote, I think this is mostly right. LLMs created a whole new layer of abstraction and profession. I've so far called this role prompt engineering, but agree it is misleading. It's not just prompting alone. There's a lot of glue code and infrastructure around it. Maybe AI engineer is usable, although it takes something a bit too specific and makes it a bit too broad. Machine learning people train algorithms and networks usually from scratch, usually at lower capability. LLM training is becoming sufficiently different for machine learning because of its systems heavy workloads

and is also splitting off into a new kind of role, focused on very large-scale training of transformers on supercomputers. In numbers, there's probably going to be significantly more AI engineers, than there are ML engineers and LLM engineers. One can be quite successful in this role without ever training anything. Okay, so let's take a moment to pause here. We talked first about how non-coders can use a new set of AI power tools to actually start to write software that fills in specific gaps in their software stack at work or that solve specific problems that they need solving.

We've talked about how there is this new category of engineer, which is specifically about the implementation and the productization of AI software and putting it into practice. Well, Rachel Woods is exploring something that, It certainly feels related but is ultimately different than the AI engineering role, which she has been calling AI operations. A good way to get into this might be to read her pinned post on Twitter right now, where she writes, for the last few months, I become obsessed with the problem of how do you build an

AI powered business, aka how do we actually use this stuff? It honestly started because at the beginning of the year, I did hundreds of AI advising calls with businesses. And while I was able to answer questions and help point them in the right direction, the meaty question they really needed answered was, how do I use AI to power my business? Pre-chat GPT, this wasn't really a question on people's radar, much less one that had good answers. Underneath the question are all sorts of sub-questions. What should people learn?

Who should businesses hire? Where should businesses start? Why do certain things? Why not do certain things? What ethical guardrails should exist? And more? I think we're still at day one in answering these questions, but I'm also proud of the progress. It's early for AI operations, but definition breeds clearer investment, focus, and is an

important step for getting real-world businesses value out of AI. Now, in a post on her Discord, she expanded out what this definition of AI operations actually means. Rachel writes, from what I've seen across what businesses need, will need, where the field is headed, and where the talent gap lies, there's a need for a holistic definition for people who are able to do a few things, drive AI literacy, implement AI solutions, and manage those efforts. What I get most excited about as someone who has worked on cutting-edge AI research teams is that I don't actually think the traditional ML engineer PhD is the right archetype for this. I think those people need people who are grounded in real-world business

problems and who can speak enough of the same language to usher solutions into organizations. Here Rachel describes that a little bit further, talking about how the idea of AI operations is putting AI into the systems that businesses run on. This is a follow-up piece of advice to the video where I shared the story of a founder I talked to who is literally telling their team they will get managed out of the company if they don't learn chat GPT by a certain date. So the flip side of this is if you want to be extremely sought after in this next AI wave, don't stop at learning how to do prompt engineering. The skill set I'm seeing every company

which they had and be willing to pay pretty high salaries or consulting rates for is AI operations, which is basically being able to take prompts. Yes, you still need to learn how to do prompt engineering, but put that into systems, processes, and automations that can be used to help a business really truly scale their operations. While prompt engineering is probably not a sustainable career, I really think AI operations will be. Now, really, really important. importantly, and I want to make this point crisply, this isn't just the prompt engineering that you heard so much buzz about a few months ago. As Rachel again tweets, underrated skills in AI ops, system design, task design, data set design. Prompt design is important, but get good at these and you can start solving very interesting operations problems. Now, what's really interesting is that one of the big themes we've been talking about for the last couple days and frankly, the last couple weeks, is the extent to which the theme of this fall when it comes to AI seems to be integration, application, actually,

putting these tools into settings and circumstances where people and businesses can use them. The AI ops role that Rachel is talking about is effectively the traditional businesses' world's bridge between the old world that didn't have these AI-powered tools and the new world in which AI-powered software and systems will underlie so much of what happens in very traditional businesses. Someone, some type of person more specifically, is going to be needed to help companies figure that out. And so here we are less than a year into the post-chatchapT world And the broad brushstrokes and outlines of these new types of roles, both for people who are engineers and for people who are not engineers, are starting to come into view. One of the things that makes this space so exciting right now is that despite the fact that hundreds of millions of people have tried these tools,

a tiny fraction of those folks have actually figured out how to integrate them into their workflows, and even fewer have figured out how to take those skills that they're developing in real time by trying and bring them into the companies that they work with or for. There is going to be so much opportunity for individuals who can help translate this new AI world to clients, to customers, to the businesses that they work with. And I think that starting to understand these categories of roles might help people frame that work just a little bit more. Anyways, guys, hope this was an interesting and useful piece. I'll include links to everything I talked about here. Big up to Swix for his work on popularizing this idea of the AI engineer. And actually, congrats to him.

He has a summit going on in San Francisco right now with the other folks from latent space, called, of course, the AI Engineer Summit, and Rachel as well as working on helping people actually put this into practice. You can check it out at theAIexchange.com. For now, though, that is where we are going to lead things for today's AI breakdown. I appreciate you guys listening or watching as always. And until next time, peace.