The AI Daily Brief: Artificial Intelligence News and Analysis - Amazon Gets In On AI Foundation Model Game with Nova

Episode Date: December 5, 2024

Amazon makes a major leap into the AI foundation model competition with the launch of Nova. Announced at AWS re:Invent, the Nova lineup includes text and multimodal models designed for speed, cost-efficiency, and advanced reasoning tasks. Also explored are Amazon's advancements in training chips and the creation of a massive new AI supercluster, Rainier, built with Anthropic. Brought to you by: Vanta - Simplify compliance - ⁠⁠⁠⁠⁠⁠⁠https://vanta.com/nlw RocketMoney - https://rocketmoney.com/aibreakdown The AI Daily Brief helps you understand the most important news and discussions in AI. Subscribe to the podcast version of The AI Daily Brief wherever you listen: https://pod.link/1680633614 Subscribe to the newsletter: https://aidailybrief.beehiiv.com/ Join our Discord: https://bit.ly/aibreakdown

Today on the AI Daily Brief, Amazon gets in the foundation model game. And before that on the headlines, Google's VO rolls out to more customers. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. To join the conversation, follow the Discord link in our show notes. Welcome back to the AI Daily Brief Headlines edition, all the daily AI news you need in around five minutes. One of the big themes we thought that 2024 was going to be all about was video generation. And to some extent, it has been. The year kicked off with a preview of SORA, which really really.

reset people's expectations of what could be. And while SORA never came out, at least not in a way that was broadly available, we got great updates from runway with their Gen 3, Luma Labs, Dream Machine, PICA. And now Google is rolling out their latest video and image generation models to AI vertex customers in a private preview. Called Vio and ImageGen 3 respectively, the models round out Google's Gen AI offering into a full suite. Focusing on Vio, the video model, Google says that Cora has already integrated its features into their Poe chatbot platform, while Oreos are owner Mondalais International is using it to create marketing content and collaboration with agency partners. Unvealed back in April, the model is capable of outputting six-second videos at 1080P.

Users can edit generated videos including changing camera movements. Speaking to the long wait for API access, Warren Barclay, the senior director of product management at Google Cloud, said it was all about ensuring, quote, enterprise readiness. He added, since Vio was announced, our teams have augmented, hardened, and improved the model for enterprise customers in Vertex AI. As of today, you can create high-definition videos in 720P and 169 landscape or 916 portrait aspect ratios. Similar to how we have improved capabilities of other models such as Gemini on vertex AI, we will continue to do this for VO. TechCrunch writes, VO understands VFX reasonably well from prompts and has somewhat of a grasp on physics,

including fluid dynamics. The model also supports masked editing for changes to specific regions of a video, and is technically capable of stringing together footage into longer projects. However, they also say reflecting the limitations of today's AI. objects in VO's videos disappear and reappear without much explanation or consistency, and VO often gets physics wrong. I think in many ways, the state of video generation is all about what you can use it for. You have tons of creators who are pushing the boundaries of short filmmaking,

but from a business perspective, this is a technology who's probably ready for prime time when it comes to advertising, social media, but not necessarily yet longer film content creation. Still with this update, it seems pretty clear that 2025 is going to have a lot more video generation than 2024 did. Next up, we've been following the recent FTC probe of Microsoft, and it appears that their deal with OpenAI is explicitly part of the concern. The information writes FTC officials have been asking Microsoft's rivals about the impact of its AI deals and range of products.

Specifically, the FTC is asking about Microsoft's deal with OpenAI. The FTC has questioned rivals about how Microsoft sells its OpenAI-infused copilot software and how it resells OpenAIs models to developers on its Azure Cloud computing platform. They continue, the question. questions imply that the FTC is probing whether Microsoft's dominance in the cloud computing market has given the company an unfair advantage in sales of AI software. Of course, this is a very fluid situation. We have a new administration coming in, and many feel that it's basically inevitable that although the next FTC chair nominee has not been announced, it is very likely that they will be less antagonistic towards big tech than Lina Khan has been.

Still, according to the information, several of Microsoft's largest rivals, quote, believe they can convince the Trump administration to keep scrutinizing the company. Adding more credence to that is, of course, the close role of advisor Elon Musk in that administration, who has been a fierce critic of Microsoft's deal with OpenAI, going so far as to actually sue Microsoft as part of his lawsuit against OpenAI as well. Speaking of OpenAI, the company has just hired its first CMO. And that person is yet another refugee from the crypto space. Kate Rauch is the outgoing CMO of Coinbase and represents the latest major hire as OpenAI

builds out its C-suite. Sarah Fryer is in as CFO, Kevin Wheel is chief business officer, although as of yet they have not hired a CTO to replace Mira Muradi. Kate tweets could not be more excited to help show the world what AI that benefits all of humanity looks like. Congrats to Kate and Good Scoop OpenAI. For now though, we've got a slightly longer than normal main episode, so we are going to cut the headlines here. Appreciate you listening as always, and now it's time for the main episode. Today's episode is brought to you by Rocket Money. We are coming up on the beginning of the new year, and that is a perfect time to get organized,

set goals, prioritize what matters most, which for many of us is going to be financial wellness. Thanks to Rocket Money, those goals, especially around money, feel achievable. Rocket Money shows you all of your subscriptions right in one place, helping you easily cancel those that you've maybe forgotten that you're actually paying for. Rocket Money also pulls together all of your spending across your different accounts so that you can clearly track spending habits and see where you can cut back. Rocket Money is a personal finance app that helps find and cancel unwanted subscriptions, monitors your spending, and helps lower your bills so you can grow your savings. Their dashboard gives you a clear view of your expenses across all of your accounts.

You can easily create a personalized budget with custom categories. You can see your monthly spending trends in each category to know exactly where your money is going. Rocket Money will even try to negotiate lower bills for you. They automatically scan your bills to find opportunities to save, and then you can ask them to negotiate. They'll deal with customer service so that you don't have to. Rocket Money has over 5 million users and has saved a total of 500 million in canceled subscriptions, saving members up to $740 a year when using all of the app's premium features. Cancel your unwanted subscriptions and reach your financial goals faster with Rocket Money.

Go to rocketmoney.com slash AI breakdown today. That's rocketmoney.com slash AI breakdown. Today's episode is brought to you by Vanta. Whether you're starting or scaling your company's security program, demonstrating top-notch security practices, and establishing trust is more important than ever. Vanta automates compliance for ISO-2701, SOC-2, GDPR, and leading AI frameworks like ISO-42,1,

and NIST AI Risk Management framework, saving you time and money while helping you build customer trust. Plus, you can streamline security reviews by automating questionnaires and demonstrating your security posture with a customer-facing trust center all powered by Vanta AI. Over 8,000 global companies like Langchain, Lila AI, and Factory AI use Vanta to demonstrate AI trust and prove security in real time. Learn more at vanta.com slash NLW.

That's vanta.com slash NLW. Today's episode is brought to you, as always, by Superintelligent. Have you ever wanted an AI daily brief but totally focused on how AI relates to your company? Is your company struggling with AI adoption, either because you're, you're getting stalled figuring out what use cases will drive value or because the AI transformation that is happening is siloated individual teams, departments, and employees, and not able to change the company as a whole. Super Intelligent has developed a new custom internal podcast product that inspires your teams by sharing the best AI use cases from inside and outside your company.

Think of it as an AI daily brief, but just for your company's AI use cases. If you'd like to learn more, go to Bsuper.a.i slash partner and fill out the information request form. I am really excited about this product, so I will personally get right back to you. Again, that's B-Super.a.ai slash partner. Welcome back to the AI Daily Brief. Amazon has had a long and interesting journey when it comes to their relationship with foundation models in the AI space. We've had context to discuss them a couple times in the last week and noted that much of their strategy was formed in reaction to chat GPT coming out and being frankly much better than the version of that that they had planned on releasing themselves.

In fact, they took the name that they had been planning to use on their chat GPT equivalent, which was Bedrock, and turned that into an AWS service to help enterprise customers figure out which models to use. Since then, they've doubled down on their relationship with Anthropic, as well as really focused on their infrastructure play with their Traneum chips. But it appears that they are not content to not be in the foundation model game, as at AWS reinvent this year, the biggest announcement of that event was the unveiling of a new family of proprietary models. Called Nova, the range includes four sizes of LLM, Micro, Light, Pro, and Premier. Nova Micro is a text-only model described as being optimized for speed and cost.

Nova Light is a low-cost multimodal model capable of quickly analyzing image video and text inputs. Nova Pro is described as a highly capable multimodal model with the best combination of accuracy, speed, and cost for a wide range of tasks. And Nova Premier is Amazon's most capable multimodal model designed to excel at, quote, complex reasoning tasks, and for use is the best teacher for distilling custom models. The lineup also includes an image generation model Nova Canvas and a video generation model Nova Real. Each is claimed to be state of the art in their respective fields. All of the models other than Nova Premier are now available within the Bedrock Model Library on AWS, with Premier

expected to arrive sometime early next year. Amazon CEO Andy Jassy said, we've continued to work on our own frontier models and those frontier models have made a tremendous amount of progress over the last four to five months, and we figured, if we're finding value out of them, you would probably find value out of them. We don't know how many parameters these models are, but based on their descriptions, they seem to line up with similar latest generation model families from the leading labs. Context windows are also comparable to rival models, but Amazon promised to deliver an ultra-long two-million token context window for some models next year. The LLMs all support fine-tuning using text images and videos as well as model

distillation. Canvas, the image model, looks on par with leading models from rival labs. While real, the video model feels a bit like a teaser version, only supporting six-second videos, which take about three minutes to generate. Amazon says that a version that can generate two-minute videos is coming soon. Quality appears up to standard in the demo video provided by Amazon. However, the user-generated videos that have come out so far range dramatically from good enough to extremely janky. Coming next in the Nova lineup is a speech-to-speech model expected in Q1 of next year and in any-to-any model expected in mid-2020. The benchmarks appear at first glance to be competitive.

Amazon is claiming that real outperformed runways Gen 3 Alpha in AB testing, achieving a 61% win rate for video quality and a 71% win rate for video consistency. For the language models, Nova Pro seems to be at least competitive with Claude 3.5 Sonnet and GBT40, claiming outperformance in some areas. One of the things that I think is important when we talk about benchmarks is that it's probably more valuable to understand these in terms of competitive ranges rather than in terms of exact specifics. And if that's how you take it, Nova Pro is in the class of models that include Claude 3.5 Sonnet, GBT, 40, etc. Wale Aachen Federer, a tech lead at AWS pointed out an

interesting area of outperformance, commenting, one exciting aspect of the newly released Amazon NovaGen AI models is their impressive performance on agent and multi-agent benchmarks. Offsetting this point, though, one of the areas that Nova Pro seems to be lacking in is encoding benchmarks. AI entrepreneur Bindu Reddy ran her own testing on live bench, finding Nova Pro was further down the leaderboard. She writes, the benefit of having a benchmark that changes every month is that it can't be gamed. On our latest November challenge, Amazon's Nova scores below Lama 70B and is slightly better than Haiku. Net net, this model doesn't change the leaderboard significantly, though it seems surprisingly fast. predictably, son at 3.5 and the O1 line remain on top of the list.

One of the more important features, though, which was kind of glossed over, frankly, at ReInvent, was how competitive Nova is on price. Nova Micro and Nova Light are both priced below Gemini 1.5 Flash and GPT-40 Mini, making them the cheapest models available from a major lab. Nova Light also has the distinction as the cheapest available model with multimodal inputs. Those two models are priced so close together that the differentiation seems to be speed, with Nova Micro about 10% faster than Gemini 1.5 Flash. Nova Light runs at a similar pace to GPT-40, which is still faster than the rest of the pack.

Nova Pro is available at around a third of the cost of either GPT-40 or Claude 3.5 Sonnet, and also positioned is slightly cheaper than Claude 3.5 haiku with much higher performance scores. The net of all of this is that Amazon's in-house LLMs are now the cheapest available for many use cases. Jerry Liu, the CEO of Lama Index, writes, Amazon's Nova should have advertised the cost reduction front and center instead of me having to dig up a hacker news thread. It's a huge value proposition. Nova is an exciting push towards way cheaper models that are comparable to the state of the art in terms of context window, performance, and multimodality.

LLMs have come a long way, but a huge issue with using them for repeated loops as cost, especially for multimodal agentic flows. Overall, the biggest takeaway from this announcement, if you're just trying to understand it in a sentence or two, is that Amazon has gone from skipping this generation of LLMs to, frankly, fully in play with a complete lineup of competitive models. Warren Professor Ethan Malick summed it up, and then there were six, or so. Based on the stats, it looks like Amazon's Nova Pro is a competitive frontier model.

This rounds out the GPT4 Gen 1 models. GPD40, Gemini 1.5, Clod 3.5, Grockt, Lama 3.2, and maybe three non-US models, Kwen, Yi, and Mistral. Another question for me is that given that Jassy is talking about making progress over just the last four to five months, it sort of implies that they've caught up to the other lab since summer, which, if that's the case, also lends credence to the growing sense that maybe there really aren't particularly strong modes when it comes to models. The next big group of announcements related to Amazon's Traneum AI chips, Traynium-2 instances are now

generally available on AWS for training and inference. The company also announced a new generation of chips Traynium 3 expected to become available late next year. For the Traynium 2 chips, Amazon is claiming a 4x improvement in speed over the first generation, which frankly saw very little adoption. They boast that Traymium 2 inference can deliver 3x higher token generation throughput on Metaslama 405B model compared to offerings available from other cloud providers. In their announcement post, Amazon claims, quote,

Traymium 2 offers 30 to 40% better price performance than the current generation of GPU-based EC2 instances, presumably referring to Nvidia's H-100s. AWS CEO Matt Garmin claimed that Traneum 3 will be twice as fast as the second generation and deliver 40% more energy efficiency. Garmin said, today there's really only one choice on the GPU side and it's just Nvidia. We think that customers would appreciate having multiple choices. One big surprise regarding Traynium was a special appearance from Apple's senior director of machine learning and AI, Benoit Dupin. Dupin took the stage to promote

Traym, stating that Apple is currently using the chips to power services like search and will evaluate the use of the latest generation to pre-train their Apple intelligence models. It's pretty unheard of for Apple to endorse a supplier, particularly in a segment as competitive as cloud. And so people are taking this as a pretty strong vote of confidence. Amazon also announced the gigantic new AI training supercomputer dubbed Rainier. Designed in collaboration with Anthropic, Amazon says the supercluster will deliver 5x the compute that was used for Anthropics' latest training runs. The design is somewhat unique with the cluster spread out across multiple facilities network together. Usually training superclusters are housed under one roof as the latency to transport data across the network is a major bottleneck.

Amazon claims to have solved this problem with networking technology that they call the elastic fabric adapter. Amazon said in their press release, When completed, it is expected to be the world's largest AI compute cluster reported to date. The current high watermark for completed supercomputers is Elon Musk's Colossus facility, which houses 100,000 Nvidia H-100s. Gotti Hut who works with customers at Amazon's chipmaking unit, Anna Perna Lab said that Rainier will contain, quote, significantly more than 100,000 Traneum 2 chips. The cluster is expected to be ready sometime next year.

So that is the story. Lots going on on the Traneum side. but the big news is Nova and Amazon being in the foundation model game for real. For now, that's going to do it for today's AI Daily Brief. Appreciate you listening or watching, as always. And until next time, peace.