The AI Daily Brief: Artificial Intelligence News and Analysis - The 10 Most Interesting and Useful ChatGPT Plugins So Far

Episode Date: May 17, 2023

From Youtube video summarizers to Spotify playlist generators to tools to chat with your PDFs, some of the most interesting and useful ChatGPT plugins as they get rolled out in Beta to all ChatGPT Plus customers. The AI Breakdown helps you understand the most important news and discussions in AI.  Subscribe to The AI Breakdown newsletter: https://theaibreakdown.beehiiv.com/subscribe Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@TheAIBreakdown

Today on the AI breakdown, we are discussing the most interesting, relevant, important, useful chat GPT plugin so far as OpenAI rolls out plugin access to all of their plus customers. On the brief, we discussed Apple's new text-to-speech features, Elon being mad at OpenAI, and much, much more. The AI breakdown is a daily podcast and video about the most important news and discussions in AI. If you're enjoying it, please like, share, and subscribe. Welcome back to the AI breakdown, all the AI headline news you need in five minutes or less. First up, Zoom has announced a partnership with Anthropic. The goal of the partnership is to bring the Claude Chatbot across the Zoom productivity platform, which will start with Zoom's contact center product being used to help with customer support.

Now, it's not clear yet how Claude might be integrated to other parts of the Zoom platform, including team chat, meetings, phone, whiteboard, or Zoom IQ. But one of the things that was notable about the announcement is that Zoom specifically called out Anthropical, constitutional AI model as one of the reasons for the partnership. They write, Anthropics constitutional AI model is primed to provide safe and responsible integrations for our next generation innovations, beginning with the Zoom Contact Center portfolio.

Maybe a vote of confidence for that constitutional AI approach. Next up, Elon Musk is not happy with OpenAI. In a recent interview on CNBC, he claimed he's the reason that OpenAI exists, that he was the one who invented the name, that OpenAI wasn't putting enough emphasis on safe AI development, and that separately he had also suffered ruptures in his friendship with Google co-founder Larry Page, because Page as well, quote, did not seem concerned about AI safety.

Musk said, quote, the final straw was Larry calling me a speciesist for being pro-human consciousness instead of machine consciousness. Now, some might feel that Elon is just angry because he's behind OpenAI and doesn't really have a stake at any of these generative AI companies, but he's far from the only one who's upset about OpenAI's moat after yesterday's hearing. One of the big threads of conversation on Twitter has been around how, Sam Altman's support for AI licenses vaguely defined at yesterday's hearing was really just another version of regulatory capture and pulling the ladder up after them.

The accusation is that OpenAI was allowed to build all these technologies without any sort of licensing regime, but now wants one just because it's out ahead. I think there's a lot more to explore here, and I probably will in a separate video later this week, but for now, that has been a big part of the sentiment on AI Twitter. Speaking of Elon and Tesla, yesterday they dropped this video of their optimist bots walking around and learning about the real world, as they put it. The way that they frame Optimus is, quote, a general purpose bipedal humanoid robot capable of performing tasks that are unsafe, repetitive, or boring. As many people have gotten excited about AI over the last few months,

some are wondering if AI powered robotics is the next big trend. Along those lines, we also got an update yesterday from a company called sanctuary where they introduced Phoenix, which is their, quote, revolutionary humanoid general purpose robot designed for work. They say Phoenix is the first humanoid to be powered by carbon, a pioneering AI control system, and represents a giant leap forward in our mission to create the world's first human-like intelligence in general-purpose robots. From sci-fi to real life. Now, one of the big themes of yesterday's hearing was concern around what was real and what was not in a world where chat GPT could communicate as well as humans. Along those lines, a Reddit post

is making the rounds in which a Texas A&M commerce professor failed their entire class of seniors, blocking them from graduating because he claimed they all use chat, GPD. Basically, he took all of their responses to a question, put them into chat GPT, and asked if chat GPT had written them, which had said it did. Now, of course, the ironic thing about this story is that what the professor accused them of doing, i.e. overly relying on AI, is exactly what the professor did, relying on an AI system that wasn't designed to detect other AI writing to tell him whether he was reading AI writing. Last up today, Apple gave us a sneak peek of some of its upcoming AI-driven accessibility updates, and that includes a text-to-speech module that can actually mimic the user's

voice. The feature is called live speech, and here's how it's described in Gizmodo. Apple is adding the ability for users to record their speech patterns a way to, in the company's words, create a voice that sounds like them for when they are at risk of eventually losing their speech. The personal voice app asks users to read a random assortment of text prompts equaling about 15 minutes of audio. The system uses AI to then generate speech that's equivalent to your personal style of speaking. Now, Apple says this will be fine because it will only be available locally on a user's iPhone, but it's still bringing up a lot of privacy and security questions as voice-mimicking technology becomes more commonplace. What do you think? Do you want an AI that sounds like you? Let me know in the comments. That's it for today's AI breakdown brief.

See you back here soon for the AI breakdown. ChatGPT is rolling out plug-in access this week to all of its plus users. So today, on the AI breakdown, we're looking at some of the most. interesting and productivity enhancing use cases that we've seen so far. Welcome back to the AI breakdown. If you have been around chat GPT and particularly discussions of chat GPT versus Google Bard, you will have heard the word plugins a lot. So plugins are effectively a set of third party in most cases applications that allow chat GPT or Google Bard to do different sets of things. So for example, an Instacart plugin would allow you to interact with the Instacart grocery shopping app. A kayak plugin would allow you to potentially search for flights, compare flights, etc.

They are a utility layer on top of an AI chatbot like ChatGBTGBT or Google Bard. Now, ChatGPT has been testing plugins in Alpha for a month or so now, but last week as part of Google I.O., Google announced that their Bard plugins would be available to everyone. Following suit, this week, ChatGPT started rolling out plugins to all of their Plus users. So today what we're going to do is look at some of the most interesting and novel use cases for these plugins and figure out what's really useful, how people are exploring them, whether they live up to the hype, and what they suggest about the future of ChatGPT. Let's first of all do a quick demo for how to actually get access to these plugins if you are a ChatGPT Plus user. So down here, you're going to want to go to your settings, and then under the Medinue over here, you'll see beta features. Now, this should theoretically have both the browser feature and the plugins feature.

For some reason, some people only have one or the other. As you can see right now, I only have plugins. But anyways, you're going to want to toggle on those features so you can actually go use them. Now, when you come back out to the main interface, if you toggle over to GPT4, you'll see down below plugins. You click that and no plugins are enabled for me, so let's go to the Plugins Store. Now, the Plugins Store looks like a basic web-based interface version of an App Store. And indeed, many people have compared the state of plugins currently to the App Store when it had less than 100 apps or something like that.

You can select most popular and see that it's things that people use pretty frequently. Open Table, Instacart, Expedia, Kayak, I think that makes sense. Or you can go to all plugins and just start going through page by page. And it doesn't take long to get a sense of, what people are building so far. There is a lot of utility plugins. For example, Voxcript here enables searching of YouTube transcripts, financial data sources and Google search results. There's a lot of summarization. World News summarizes news headlines. You can ask for the latest news from various sources around the world. Research assistants like Yabble, your ultimate AI research assistant,

create surveys, specify audiences, collect data and analyze. A lot of emerging financial applications like Publix's real-time and historical market data. Tons of integrated shopping. And of course, just some random cool things like Playlist AI, which says create Spotify playlist for any prompt. Let's do that one as the demo just because it sounds pretty interesting. Here's the terms. They access my Spotify account. I'm already logged in. Okay. So let's try, make me an energetic work playlist with Hyperpop and Synthwave tracks. at least 10 tracks, no repeats. Let's see what it does.

100 gecks, we're on the right track. And then you can actually see it goes a step farther than just putting the tracks here by creating the playlist using this plugin. So now it's complete and let's see how it did. Boom, there it is. Some fairly good choices. We got Nightcall, a classic.

We got some 100 geeks. Not so bad. Okay, so now you get the idea of how plugins work. Like I said, they are a utility layer on top of chat GPT. and we'll come back to why that's such a different use case in a few minutes. But before that, let's talk about some of the plugins that people are using and how they are using them. Now, for these, rather than go through all of them on my own chat GPT,

I'm going to use examples from Twitter because I think people have already done the hard work of summing up some of the most interesting use cases. The first that we're going to talk about is golden. And the reason that I wanted to put it first is that it comes out a problem that I think is going to be relevant for all of these other plugins, which is accuracy of AI information. Jude Gomilla writes, Without the plug-in, chat GPT knowledge is pulled from its LLM, which on the pre-trained model cuts off at mid-20201.

The golden plugin gets data that is current, cited, and directly improvable if missing and incorrect. No more black box for our facts. The plugin uses retrieval enhancement where we fetch the latest data from the golden knowledge graph and infuse into chat GPT as source knowledge to produce citable facts about entities and natural language conversation. We always want to know where information came from.

For example, asking the very basic question. of who is the CEO of Twitter to GPT4 results in the outdated answer of Jack Dorsey, but with enhancement results is the correct answer of Elon. Editor's note, he posted this before Linda Yacaro was made the CEO of Twitter. Jude sums up, chat GPT gets really useful when everything is up to date. So like I said, I think that this type of information accuracy and information sourcing is going to be a big part of how AI grows and develops and becomes more trusted in society, so I thought I wanted to call that one out.

Next up, let's turn to a super basic day-to-day kind of use case, which comes from Amar Reshi, who's a design manager at Brex. He says, I asked ChatchipT to buy my groceries today using Instacart's plugin, and it works so well. Stayed within my budget, provided ingredients and recipes for seven meals, accounted for my schedule and diet. Then in this Twitter thread, he shows the conversation, and there are a couple things that elevate this, I believe, from more than just a novelty. First of all, he's effectively outsourcing thinking about what he wants to eat to Chat Cheap-C-T, and given that humans basically spend most of their time thinking about what they're going to eat next,

that's actually fairly time-saving. Second, ChatGPT with the help of the plugin is able to go from ideas for meals to the actual recipes and ingredients in short order, again, saving more time. Finally, it pushes directly to Instacart so he can press Shop on that app. Now, here it's worth noting that right now there isn't commerce integrated to ChatchipT, so he's still being pushed offsite to Instacart.com to his account to actually make the order. Next up, another time-saving, personal assistant day-to-day type use case that shows off another capability of the new plugin system. So this one comes from Data Chaz and he writes,

Another thing I like about ChatGPT's plugin system is its ability to select the most appropriate plugin based on the user's query. In this demo, even though I have multiple plugins enabled, Chad ChpT correctly chose to use the kayak plugin, which is exactly what I wanted. His prompt is book a trip for two from Paris to Venice under 300 euro, show me the best price. from July to August 2023. ChatGPT gets right into it, and again, what he pointed out is that it actually figured out that although he had multiple prompts that were available, kayak was the right one to use. So this is not only an example of travel planning, which is a use case for these plugins, but also the fact that the system is sophisticated to be able to help you navigate which

plug-in to use in any given circumstance. Next up is a use case that it seems like a lot of companies are exploring, which is financial research. In this particular case, the plugin being demoed is portfolio pilot, which gives research on public companies, but we also saw just in the quick browse that we did of the plug-in store, a number of other financial services plugins like public as well. I'm planning on a whole video soon about these financial use cases of chat GPT and why they might be ones that we should be, if not concerned about, certainly more conscientious of the potential for misinformation around. But what they definitely represent is the idea that plugins are going to enable highly specialized research, where they hone in chat GPT to a particular

research use case and give chat GPT more information to execute against that research effectively. Let's move into the realm of productivity. One of the things that is a major use case for these types of chat bots that we're seeing come up more and more is effectively chatting with PDF. So here, sure enough, is a plugin called chat with PDF. Chat analyzed question with any PDF directly in the comfort of chat GPT via plugins access. Here it is helping me with homework. He then gives ChatGPT a URL of a PDF and it starts summarizing it right. away. Now there are a number of plugins and standalone tools for exactly this, so it seems like a use case that people are really excited about. Speaking of summarization, about a month ago,

YK, who goes by CS Dojo on YouTube, writes, I started building a chat GPT plugin that can summarize any YouTube video, answer questions about it, and give specific timestamps when asked. This again speaks to that productivity enhancement use case where we're taking big volumes of information, ingesting them, and then spitting out summaries that can help people understand key details in shorter order. I think the idea of giving specific timestamps where you can go back and review key information in the video makes this extra useful. Now, when it was shared by YK, this plugin was still in development. It wasn't necessarily in the store. And when I went back and looked in the plugin store, I didn't find it exactly, but I did find something called

Video Insights and went to see if it did something similar. So I asked ChatGPT, can you summarize this video when it was my video yesterday about seven takeaways from the Senate's AI hearing? and it came back with what was in the description and the timestamps. In fact, it said, unfortunately, I can't provide a detailed summary because the video is longer than 10 minutes. However, based on the description, it seems to focus on the Senate's hearing on AI and the various perspectives and concerns raised during the discussion. So while this might not be exactly what you're looking for with this type of summarization, it certainly shows the possibilities moving forward. Next up, one of the plugins that people are really excited about is called perfect prompt. And what this does is it makes your prompts better.

Given how much of being successful with generative AI systems is being able to prompt well, this type of tool that can go from generic to very specific and useful prompt is something that people are really, really interested in. In this example from Min Choy, he writes, perfect 20 AI news trending with URL links. Perfect prompt suggests a rephrasing of what are the top 20 news stories related to artificial intelligence that are currently trending, and can you provide URL links to each of them? Now, this also uses WebPilot, which is a plugin that allows ChatGPT to browse. And of course, that Internet access is a huge part of what makes these plug-ins so much more useful than just ChatGPT alone.

Now, one plugin that we've talked about before actually isn't a third-party plugin at all, but is one of the two plugins that OpenAI has worked on itself. And that, of course, is Code Interpreter. As Peter Yang puts it really well here, code interpreter gives everyone a personal data analyst. So here's Peter's example. He writes, I uploaded an Excel file, and without prime, prompting it identified the source World Happiness Report 2021, columns and definitions. Now let's give it the incredibly lazy prompt, analyze this data, and show me cool charts. Here are some charts that Code Interpreter came up with.

Top 10 countries by Happiness Score, Happiness Score by Region, Happiness Score via GDP per capita, Happiness Score via Life Expectancy. This all took less than a minute to generate. Next, I uploaded two separate datasets to see if it can do an analysis that combines the data. And it works. Here's Code Interpreter mapping happiness against life expectancy. by country. And you can examine the code in the chat interface to understand how it produces the charts. The biggest problem with chat GPT is that it tends to make stuff up. But code interpreter doesn't seem to hallucinate much at all because it's all based on data I actually uploaded. That's incredibly powerful. Now, if you want more on code interpreter, go check out my video from a couple weeks ago

called Sixways ChatGPT code interpreter is already being used. There are a ton of very cool examples in there. Now, we're going to close out with a couple of use cases that are a little bit more experimental. and actually both come from Pietro Sherano. Pietro writes, couldn't wait for multimodal GPT anymore, so I built my own with plugins. Introducing Picasso, a chat GPT plugin delivering multimodal-like performance.

Picasso uses five different models to achieve this. Here are some examples of extracting recipes from photos and more. So basically you have here an input of a photo of a plate of spaghetti, and chat GPT thanks to Picasso is extracting information from that picture, giving the recipe, and then flipping the flow and generating two similar photos based on those inputs. We've profiled a number of tools on this channel like Mini GPT4 for going from image to text, so it's very cool to see it built in natively to chat GPT with this plugin.

Pietro's other fascinating plugin experiment is called Designer GPT, and he writes, My new plugin that lets you create any website directly in chat GPT. It seamlessly integrates with the stable diffusion plugin, and I didn't code that. GPT just knows how to use both, absolutely unreal. Websites are hosted on a remote server via Replit. So the prompt he uses is generate a website for a beautiful architecture firm,

use any photos you want. And sure enough, what comes out is a website, basic though it is, for an architecture firm. Now, how much this is really useful versus just novelty, especially in the wake of so many AI generated website tools, who knows, but it does show off the capacity of this chat GPT plugin system. And I think that that was Pietro's goal, more as a thought starter about what can be done than necessarily getting to an end point that is super useful in and of itself. Now, it's worth remembering as we wrap up that this is all super nascent. There are less than 100 plugins in the plugin store right now, and really people are just beginning to experiment. Ethan Mollock cautions wisely before the influencer threads starts.

Plugins and web browsing are both still early stage and not miracles yet. Browsing is especially early. So be careful about the everything just changed OMG posts, but do play around with the systems. They have potential. And in that spirit, Rachel Woods writes, not a critique, but just an observation. She says, after spending a big chunk of the weekend playing with all the new chat GPT plugins, I have a few observations that are shaping what seems to be. be ahead for the Everything app, aka ChatGPT.

I've found that plugin usage feels more utility focused. I need to get X done rather than exploration or like some of the meandering conversations I have with ChatGPT itself. This could be due to needing to plan ahead and choose which plugins you'll use for a thread. Oftentimes I open up a chat thread not really thinking about where it'll end up. The current ChatGPT plugin UX forces pre-planning. Second, the plugins are fickle, like really fickle. If you're not familiar, the way plugins work is by giving chat.

GPD and API in a manifest or a natural language explanation of how to use the plugin. If you thought hallucinations were a major limitation, the variability of plugins working or not feels even bigger. Third, it blurs the lines even more on copyright, data privacy, and ownership. A third party could collect a bunch of my content to power a plugin and doesn't cite their sources when used. Is that okay? If plugin developers will eventually need some usage data on how their plugin is being requested, say to optimize their manifest file, how much data will they get? The whole prompt or the whole thread? If you've forgotten how early we are in this AI wave, go play with chat GPT plugins.

You'll probably quickly remember. I think these are great observations. It does absolutely feel like a utility layer on chat GPT versus an extension of this sort of magic of meandering through different discussions. Now, for a lot of use cases, that's much better. But it does also create the types of issues that Rachel brings up. In any case, it is an exciting development for sure. It is making chat GPT a whole different level of useful for lots and lots of different use cases.

So go explore and let me know in the comments what you're using it for, what you found good, what you found didn't work as well, and we'll check back in in a couple weeks to see if some use cases and plugins are racing out into the lead. That's it for this AI breakdown. If you were enjoying these, please like, subscribe and share it. Go listen to the podcast version or subscribe to the AI breakdown newsletter. I appreciate you watching or listening, and until next time, peace.