The AI Daily Brief: Artificial Intelligence News and Analysis - AI is the First UI Paradigm Shift in 60 Years

Episode Date: June 22, 2023

AI represents the third major UI paradigm shift in computing. From batch processing to command-based interaction to intent-based outcome specification. A reading of AI: First New UI Paradigm in 60 Years by Jakob Nielsen   Before that on The Brief Marvel has kicked up significant controversy around the AI generated title sequence for the new Secret Invasion show on Disney+; Senate Majority Leader Chuck Schumer announced SAFE Innovation framework; Wimbledon launches AI-powered commentary, and Dropbox starts a $50M AI venture fund.   The AI Breakdown helps you understand the most important news and discussions in AI.  Subscribe to The AI Breakdown newsletter: https://theaibreakdown.beehiiv.com/subscribe Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@TheAIBreakdown Join the community: bit.ly/aibreakdown Learn more: http://breakdown.network/

Today on the AI breakdown, we're talking about how AI represents the first UI paradigm change in 60 years. Before that on the brief, Chuck Schumer lays out a new policy framework, Marvel court's controversy, and much, much more. The AI breakdown is a daily podcast and video about the most important news and discussions in AI. Like, subscribe and share, and go to Breakdown.network for more information. Welcome back to the AI breakdown brief, all the AI headline news you need in five minutes or less. We kick off today with what is got to be just the beginning of controversies that relate to AI being used in popular media. Gizmodo writes, Secret Invasion uses AI for its opening credits and the excuse is meaningless.

The new Marvel Disney Plus show crosses a messy line for AI-generated works presence in mainstream media. Moto writes, If you hopped on Disney Plus this morning to see what Nick Fury and his frenemies were up to in Marvel's Secret Invasion, you may have noticed something a little off about the series opening title sequence. It looks strange to your mind's eye and for good reason. It's one of the first major shows to use AI-generated art in such a way. The story goes on to explain that the titles were developed by Method, who had also worked on previous Marvel shows, including Moon Knight, Miss Marvel, and Loki,

and that with this particular task, they were, in fact, approached to use AI-generated imagery. This wasn't just something they turned in, it was a plan all along. Gizmodo writes, That hazy surreality was the intention behind using AI-generated imagery. Producer Ali Salim said, when we reached out to the AI vendors that was part of it. It just came right out of the shape-shifting scroll world identity, you know? Who did this? Who is this?

Selim went on saying that he didn't really understand how AI worked, but, as Polygon put it, was fascinated with the ways in which the AI could translate the sense of foreboding he wanted for the series. Salim said, quote, we would talk to them about ideas and themes and words, and then the computer would go off and do something, and then we could change it a little bit by using words and it would change. Gizmodo's complaint seems to be about AI's use in major media in general. They write, in a world where creatives have repeatedly pushed back against commercial usage of tools like Dali and Mid Journey, citing concerns of displacing jobs for human artists, plagiarism from databases trained on uncredited stolen artwork, ethical crediting practices, both in the worlds of art and cinema in general, as the ongoing writer strike has made AI content protections a key part of union demands. Secret Invasion's title sequence is a scary line to cross.

The Gizmodo piece ends. It felt explorative and inevitable and exciting indifference, Salim concludes while talking to Polygon, a comment that will no doubt ring hollow and chilling to creatives concerned by AI imagery's growing presence. On Twitter, people's reaction to this was absolutely a Roar Shack test for how they feel about AI in general. Brian Long writes, So Marvel really used AI to make their intro for Secret Invasion. It's actually over. Stephen Ford writes,

Marvel and Disney have infinite money yet used AI for the Secret Invasion opening credits. A slap in the face to literally every artist Disney has ever worked with and something that overshadows the hard work everyone did on this show. Kelly McCurnan writes, Marvel used AI for the Secret Invasion intro. That was a paying job for many artists. Add this to the heap of industry-wide lost work replaced by plagiarism software, aka generative AI. Absolutely disgusting. On the other hand, some digital artists were excited. Omli Art writes, whoa, so exciting to see AI art being utilized in all its imperfect beauty on Marvel's Secret Invasion. Cudos to Method Studios. It's gorgeous.

Chris Cashinova writes, wow, an official movie sequence made with AI. And MK Toon writes, everybody take a big breath. Marvel didn't feed the secret invasion opening into a computer and spit it out on Disney Plus. Method Studios, a VFX companies that staffs human producers, human designers, and yes, human AI technicians worked on it. They used AI as a tool. So friends, this is absolutely the beginning of a type of controversy that we're going to see a lot of. And it'll be interesting to see in a couple weeks whether anyone cares anymore or whether we've just moved on to something else. Next up, a very different type of story. We're starting to get more information about how the Senate plans to regulate AI. Senate Majority Leader Chuck Schumer has outlined a new framework, if not

actual, new policies. Schumer has named the framework the safe innovation framework for artificial intelligence. In a speech, Schumer said, after months of talk with over 100 AI developers, executive, scientists, researchers, workforce experts, and advocates, this morning I'd like to share my proposed framework for action. I call it the safe innovation framework for AI policy, the safe innovation framework. I call it that because innovation must be our North Star. The U.S. has always been a leader in innovating on the greatest technologies that shape the modern world. But if people think AI innovation is not done safely, if there are not adequate guardrails in place, it will stifle or even halt innovation altogether. So it is safe innovation that we must seek. So of course, this being

Congress, that safe word has to stand for something. And in this case, it stands for security, accountability, protecting our foundations. And lastly, explainability. Now, while this is very light on actual policies, I tend to think that's a good thing. It suggests to me that they're taking this seriously and are trying to avoid any sort of knee-jerk reaction. There's clearly growing technical awareness. For example, here's how Schumer explains explainability. He says, explainability is perhaps the greatest challenge we face on AI. Even the experts don't always know why these algorithms produce the answers they do. It's a black box. No everyday user of AI will understand the complicated and ever-evolving algorithms that determine what AI systems produce

in response to a question or task. And of course, those algorithms represent the highest level of intellectual property for AI developers. Forcing companies to reveal their IP would be harmful, who would stifle innovation, and it would empower our adversaries to use them for ill. Fortunately, the average person does not need to know the inner workings of these algorithms, but we do need to require companies to develop a system where? In simple and understandable terms, users understand why the system produced a particular answer and where that answer came from.

This is very complicated work. Now, if you're asking what the next steps are, Schumer is convening a series of what he calls AI Insight forums. Quote, we need the best of the best sitting at the table. The top AI developers, executive, scientists, advocates, community leaders, workers, national security experts, all together in one room, doing years of work in a matter of months. The panels will include people of differing views, including some skeptics. Each insight forum will focus on the biggest issues in AI, including asking the right questions,

AI innovation, copyright and IP, use cases in risk management, workforce, national security, guarding against doomsday scenarios, AI's role in our social world, transparency, explainability, and alignment, and privacy and liability. Schumer says these insight forums are the first of their kind. They have to be the first of their kind because AI moves so quickly. If we take a typical path, holding congressional hearings with opening statements and each member asking questions five minutes at a time on different issues, we simply won't be able to come up with the right policies. By the time we act, AI will have evolved into something new. This is very encouraging to hear as someone who has watched a lot of congressional hearings over the last few years. Congressional hearings, as they are currently constituted, tend to just be chances for legislators to get in soundbites that they can use in future cam.

pains. So even if this new type of forum doesn't exactly work as planned, at least it's not that. Schumer concludes, no question about it, this is all exceedingly ambitious. We must exercise humility as we proceed. We are going to work very hard to come up with comprehensive legislation, because this is so important we are going to do everything we can to succeed. But success is not guaranteed. AI is unlike anything we've dealt with before, and it may be exceedingly difficult for legislation to tackle every issue. Humility is the key word. The AI revolution is going to happen with or without us. not know of any other instance in human history when we reach new heights, uncovered new truths, or mastered new innovations only for us to turn back.

It's in our nature to press ahead. So, friends, let us not turn back. Let us not look away. Instead, let us forge ahead determined and unafraid to lay a foundation for the next era of human advancement. Obviously, we will continue to follow up on the U.S. policy discussion around AI, but I will only say this. I don't know that I've ever seen another issue in my lifetime that has gone from not on the

agenda to the very top of the agenda so quickly. And that I think is actually cause for celebration. It suggests to me that politicians actually understand the magnitude of what we're dealing with. Now, we're going a little long on the brief today, but there is so much happening. And these were both stories that could have been features, but let's still try to catch up on a few of the other stories as well. You might remember a month or two ago when Dropbox announced a set of layoffs. The layoffs weren't people being replaced by AI, but they were people whose skill set wasn't a good fit with Dropbox as it reoriented itself to compete in an AI-powered world. Well, now, just a little bit later, we've started to see how Dropbox is thinking about

its approach to AI. Yesterday, they announced a number of new initiatives, including a $50 million AI venture fund, a universal search bar which can search across third-party tools, and something they're calling Dropbox AI, which summarizes and extracts info from files stored in Dropbox. Meanwhile, one company that did cut a big part of its workforce for AI is the build tabloid in Germany. Build is the biggest newspaper in Germany and has announced that it will cut about 20% of its workforce for AI. In a memo obtained by CNN, the company said that traditional roles

like editors and proofreaders will, quote, no longer exist as they do today. Next up, Global Cybersecurity Company Group IB says that they've identified over 100,000 sets of chat GPT credentials within the logs of info-stealing malware that trades on dark web marketplaces. The number of compromised accounts reached a peak in May of 26,802. So obviously, if you are using, chat GPT like any other service, please enable two-factor authentication and keep yourself safe. Moving on, Wimbledon has raised some higher announcing that they will be introducing AI-powered commentary to coverage this year. AI generated commentary and captions will be used for all of its online video highlights. Now, while the amount of AI content is limited, IBM said that the move

is just one step along the way to generating AI commentary on full matches. Finally, another milestone in the chat GPT plugin store. As of yesterday, there are now 500. available plugins. Now, of course, how many of them are useful is a subject for another video. That's going to do it for today's AI breakdown brief. If you're enjoying this, please like, subscribe and share, and I'll be back soon with the main AI breakdown. Welcome back to the AI breakdown. As I mentioned in the intro to this show, I am currently traveling. I have a few interviews coming up. There'll be an interview on Friday, an interview on Saturday, and then an interview again on Monday. But the plan for Tuesday, Wednesday,

and Thursday is to do the brief, but potentially not the main part of the episode. We'll see how the timing works, but certainly I will get you guys a brief at least. Now, today, as this show comes out, I am racing to catch a plane inevitably, and I came across an article from June 18th just a couple days ago that I thought was really interesting. It was written by Jacob Nielsen, who's a principal at the Nielsen-Norman Group, which focuses on research-based user experience. The piece is called AI, the first new UI paradigm in 60 years, and Jacob summarizes, AI is introducing the third user interface paradigm in computer history,

shifting to a new interaction mechanism where users tell the computer what they want, not how to do it, thus reversing the locus of control. On the one hand, this is the type of thing that's sort of wonky and behind the scenes, but on the other hand, this is a recognition of a big pattern in history that has shifted, and I think trying to understand that paradigm shift is really valuable. So today as I travel, you guys get a long reads Thursday about AI, The first new UI paradigm in 60 years.

ChatGPT and other AI systems are shaping up to launch the third user interface paradigm in the history of computing, the first new interaction model in more than 60 years. Section the first two paradigms. Paradigm 1. Batch Processing From the birth of computers around 1945, the first UI paradigm was batch processing. In this paradigm, users specified a complete workflow of everything they wanted the computer to do. This batch of instructions was submitted to a data center, often as a deck of punched cards,

and was processed at some unspecified time, often overnight. Later, often the next morning, users would pick up the output of their batch. Usually this would be a thick fan fold of printouts, but it could also be a new deck of punched cards. If the original batch contained even the slightest error, there would be no output, or the result would be meaningless. From a UI perspective, batch processing did not involve any back and forth between the user and the computer. The UI was a single point of contact, that batch of punched cards. Usability was horrible, and it was common to need multiple days to fine-tune the batch

to the point where executing it would produce the desired end result. Paradigm 2. Command-based interaction design. Around 1964, the advent of time-sharing, where multiple users shared a single mainframe computer through connected terminals, led to the second UI paradigm, command-based interaction. In this paradigm, the user and the computer would take turns, one command at a time. paradigm is so powerful that it has dominated computing ever since for more than 60 years. Command-based interactions have been the underlying approach throughout three generations of user interface technology. Command lines like DOS and Unix, full-screen text-based terminals

common with IBM mainframes, and graphical user interfaces, Macintosh, Windows, and all current smartphone platforms. Powerful and long-lasting, indeed. The benefit of command-based interactions compared to batch processing is clear. After each command has been executed, the user can reassess situation and modify future commands to progress toward the desired goal. In fact, users don't even need to have a fully specified goal in mind because they can adjust their approach to the problem at hand as they get more information from the computer and see the results of their initial commands. Early command line systems often didn't show the current state of the system, with horrible

usability as a result. For example, in Unix, no news was good news because you would get feedback from the computer only if your command resulted in an error message. No errors meant no information from the computer about the new state, which made it harder for users to compose the following command. The beauty of graphical user interfaces is that they do show the status after each command, at least when designed well.

The graphical user interface has dominated the U.S. world since the launch of the Macintosh in 1984, about 40 years of supremacy until it possibly is replaced by the next generation of UI technology and, more importantly, the next UI paradigm in the form of artificial intelligence. Section. The newest paradigm. Paradigm 3. Intent-based outcome specification.

I doubt that the current set of generative AI tools like ChatGBT, GBT, BART, etc., are representative of the UIs will be using in a few years, because they have deep-rooted usability problems. Their problems led to the development of a new role, the prompt engineer. Prompt engineers exist to tickle ChatGBT in the right spot, so it coughs up the right results. This new role reminds me of how we used to need specially trained query specialists to search through extensive databases of medical research or legal cases.

Then Google came along and anybody could search. The same level of usability leapfrogging is needed with these new tools. of AI should be a significant competitive advantage. And if you're considering becoming a prompt engineer, don't count on a long-lasting career. The current chat-based interaction style also suffers from requiring users to write out their problems as prose text. Based on recent literacy research, I deem it likely that half the population in rich countries is not articulate enough to get good results from one of the current AI bots.

That said, the AI user interface represents a different paradigm of the interaction between humans and computers, a paradigm that holds much promise. As I mentioned, in command-based interactions, the user issues commands to a computer one at a time, gradually producing the desired result if the design has sufficient usability to allow people to understand what commands to issue at each step. The computer is fully obedient and does exactly what it's told. The downside is that low usability often causes users to issue commands that do something different from what the users really wanted.

With the new AI systems, the user no longer tells the computer what to do. Rather, the user tells the computer what outcome they want. Thus, the third UI paradigm, represented by current generative AI, is intent-based outcome specification. A single example of a prompt for an AI system is, make me a drawing suitable for the cover of a Pulp Science fiction magazine, showing a cowboy in a space suit on an airless planet with two red moons in heaven. Try ordering Photoshop circa 2021 to do that.

Back then, you would have issued hundreds of commands to bring forth the illustration gradually. Today, Bing Image Creator made me four suggested images in a few seconds. With this new UI paradigm, represented by current generative AI, the user tells the computer the desired result, but does not specify how this outcome should be accomplished. Compared to traditional command-based interaction, this paradigm completely reverses the locus of control. I doubt we should even describe this user experience as an interaction, because there is no turn-taking or gradual progress. That said, in my science fiction illustration example, I'm not happy with the spacesuits.

This might be fixed by another round with the AI. Such rounds of gradual refinement are a form of interaction that is currently poorly supported, providing rich opportunities for usability improvements for those AI vendors who bother doing user research to discover better ways for average humans to control their systems. Do what I mean, not what I say, is a seductive UI paradigm. As mentioned, users often ordered the computer to do the wrong thing. On the other hand, assigning the locus of control entirely to the computer does have downsides, especially with current AI, which is prone to include erroneous information in its results.

When users don't know how something was done, it can be harder for them to identify or correct the problem. Intent-based paradigm doesn't rise to the level of non-command systems, which I introduced in 1993. A true non-command system doesn't require the user to specify intent because the user acts as a side effect of the user's normal actions. As an example, consider unlocking a car by pulling on the door handle. This is a non-command unlock because the user would perform the same action whether the car is locked or unlocked. In contrast, a car operated by voice recognition could unlock the door because the user stated, I want the car to be unlocked, which would be an intent-based outcome specification.

An old-fashioned car could be operated by the explicit command to unlock the door by inserting and twisting the key. Whether AI systems can achieve high usability within the intent-based outcome specification paradigm is unclear. I doubt it because I am an enthusiastic fan of graphical user interfaces. Visual information is often easier to understand and faster to interact with than text. Could you fill out a long form like a bank account application or a hotel reservation by conversing with a chatbot?

Even one is smart as the new generative AI tools? Clicking or tapping things on a screen is an intuitive and essential aspect of our use. user interaction that should not be overlooked. Thus, the second UI paradigm will survive, albeit in a less dominant role. Future AI systems will likely have a hybrid user interface that combines elements of both intent-based and command-based interfaces while still retaining many GUI elements. Again, back to NLW for just a quick wrap up here.

I really do love the simplicity of the explanation of moving from a paradigm in which we tell computers what to do to one in which we tell computers the result we want on the other side, and let them figure out how to do it. To put it in the context of a conversation which many people have had around AI in the last couple months, part of the excitement around autonomous AI agents, things like AutoGBT or Baby AGI,

is the idea that we can give an AI an objective and let it figure out, one, the steps to do it, and then two, how to execute along each of those steps. I think that's a demonstrative example of this intent-based outcome specification that Jacob is talking about. I also tend to agree on first blush that whereas command-based interaction entirely replaced batch processing,

there will remain use cases where that command-based interaction still makes sense, where GUIs still makes sense. I wouldn't be surprised, however, if many of the things that I assume I would still want to use a GUI for do in future generations get replaced by this new paradigm. But if nothing else, GUIs will still have us boomers to extend their lives for a few more decades. Anyways, guys, that is it for today's AI breakdown.

Thanks to Jacob Nielsen for writing a really thought-provoking an interesting piece. Thanks to you guys for listening. And until next time, peace.