Big Technology Podcast - GPT-6 & OpenAI’s Comeback, Hugging Face Attack Debate, Ballmer’s Scandalous Legacy

Episode Date: September 4, 2026

Ranjan Roy from Margins is back for our weekly discussion of the latest tech news. We cover: 1) OpenAI releases GPT-6 2) OpenAI President Greg Brockman says AGI is here 3) Is AGI more useful to OpenAI as something that never comes? 4) GPT-6 crushes the benchmarks 5) But where's the economic activity? 6) Are the latest models less monitorable? 7) OpenAI bots hijack a german website 8) The hugging face attack was worse than initially disclosed 9) Could agent swarms be used for good? 10) Steve Ballmer's legacy up in flames after Kawhi scandal 11) Developers, developers, developers, etc. --- Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice. Want a discount for Big Technology on Substack + Discord? Here’s 25% off for the first year: https://www.bigtechnology.com/subscribe?coupon=0843016b Learn more about your ad choices. Visit megaphone.fm/adchoices

Open AI releases GPT6 and says it's AGI. Has it retaken lead? How serious should we take the hugging face attack anyway? And Steve Balmer steps in it. That's coming up on a Big Technology Podcast Friday edition right after this. This episode is brought to you by Genesis. If you're following where Enterprise AI is actually going and want to get deeper than the headlines, you're going to want to check out Genesis Experience 2026 this September.

With two full days of free life keynotes streaming from Las Vegas on September 2nd and 3rd, You'll learn about agentic orchestration, the new operating model for customer experience, and real stories from organizations already deploying AI at scale. And I'll be there, filming interviews for our YouTube channel. Experience 2026 is where enterprise leaders learn how to turn customer experience into growth. And you can watch it all live. Register free at genesis.com slash experience slash broadcast. Two and five Canadians will hear the words you have cancer.

That's why every step and dollar raised matters. On September 19th, join thousands in Toronto for the Princess Margaret Cancer Foundation Walk. Challenge yourself, friends, and family to walk 21 kilometers in support of life-saving research. Together, we can carry the fire and help create a world free from the fear of cancer. Register today at pmcfwalk.ca. Welcome to Big Technology Podcast Friday edition, where we break down the news in our traditional cool, and nuanced format. We have a great show for you today.

We're going to talk all about OpenAI's new GPT6 model. It says it's AGI and has it finally come back and taken the lead over Anthropic? We're also going to talk about the hugging face attack. And there was also another attack, according to a new Reuters report, where bots coordinated on a German wiki website. So we're going to talk about whether this is, again, whether this is marketing or whether it's time to finally take these attacks seriously. and we're going to go a little bit into the intricacies of what happened.

And finally, Steve Balmer, the legacy does not look good. Of course, we're talking about the fact that he was directly involved in this scheme where the Clippers paid Kauai Leonard, according to the NBA, lots of money to not show up to certain jobs to subvert the NBA salary cap. What would a Labor Day weekend Friday edition look like without some Steve Balmer talking here to do it with us. As always, is Ron John Roy of margins. Ron John, great to see you. Welcome back. If Kauai Leonard was paid to not show up, our listeners, I can tell you, Alex Cantwitz is showing up because if you're watching this on YouTube, you will see a beautiful new studio

that Alex is going to be broadcasting out of. Alex, did you set this up yourself? This thing is gorgeous. That's right. Okay, so we do have a new studio. I did not set it up myself entirely. everything that you can see behind me was my doing. Everything that you can't see, which means the camera, the lighting, and all of the settings, that was done with some help. But I definitely did get the paneling from Amazon and nail them to the wall, panel by panel. And we don't even have a hammer, so I was nailing them to the wall with the back end of a wrench. So if one falls on me during the show, you understand why.

That is a New York City living toolboxing and like being a handyman. Well, we did have a hammer, but I couldn't find the hammer. And the panels needed to go up. So you got to do what you got to do. You got to do what you got to do? AI is not going to replace that. Let's just, let's just. Well, I don't know.

After the latest release from OpenAI, maybe it will. So OpenAI released GPT6 yesterday. And remember the big weight for GPT5? when's it coming and the expectations. This kind of came seemingly out of nowhere. We have GPG6 Astra, and not only does it crush on the benchmarks, opening eye, it's saying that it might be AGI,

and it's obviously geared towards a lot of sort of personal assistant use cases. So let me just read a little bit from what the Verge reported on this. So the Verge says, Opening Eye's next big AI model has entered the AGI era. The next big model is here. It's called GPT6 Astra. The company calls it a digital. generational leap in capability and capability for areas like cybersecurity, professional work,

software engineering, science, and computer use. The actual, this is, I'm just going to read from opening eyes branding here. They say it's GPT6 Astra. Anything you can do on a computer Astra can do for you fast. And they, of course, released a, you know, a snazzy video as they tend to do on these releases with showing people in front of a big computer asking it to do things for them like book tables, build a presentation, create legal drafts. And basically the idea here is that their new model is going to excel at computer use and be able to get things done for you. And it's very interesting that they use. They highlighted voice as the interface to get it done, almost like the personal assistant computer in Star Trek. So, Rhonda, your thoughts about the release of

GPT6. I'm curious to hear your initial reaction, but also like we talk a lot about like the fact that even recently I talked about how, you know, it looked like Anthropic was opening up the gap between itself and open AI. Maybe the gap has shrunk or closed completely. What do you think? All right. Let's separate out those two questions, what this means in the, the AI race. And then first, you know, like, is this an exciting launch? Again, I always, any of these new model launches, try to wait until I've actually had access to it. And I'm unfortunately not part of the daybreak platform and an open AI cybersecurity researcher. So that'll have to wait. Right. Those are the people that have gotten initial access. It's already a little controversy

because a handful of people can already use it. And it's supposed to roll out to everybody else soon, but it hasn't yet. But that will come in time. Go ahead. But it is certainly rolled out to every ex-influencer who has now built some virtual world or recreated a video game. or whatever else and has posted about that. But I do love that, like my favorite part of the launch announcement was AGI is here, build new world models, like the crushing benchmarks, but create nice decks and book at restaurant table that it still always comes back to that. I love that the test of AGI in the end is going to be, can you actually book a restaurant

table or create a good PowerPoint deck? I think, like, I don't know. do you do you have an opinion on how big this is already are you excited it's hard for me to try to gauge on that i think on the benchmark side i think it's really interesting and i think in the anthropic context it's even more interesting but on is this really exciting i don't know yet right so it's a great question and i'm kind of on two minds about it so you know the way to sort of think about these releases is, you know, I do think to some degree you can't use all of what these companies say about the releases as gospel when they come out. But you can sort of take some

signals because they are putting their reputation on the line to some degree. And, you know, earlier this year, I was at Open A.I. with Craig Brockman. And he said that he thought the company was about 80% of the way towards AGI. Very different comments with this new model. So he says, if we fast forwarded a couple years and we look back and say, when was it really? really that AGI was created, I think it's going to be about this time. I think it might be about this model. For me personally, I do think we're there. I think it's not unreasonable to feel that we are now in the AGI era. Okay, I read this and it sort of was like, you know, you know, when you want to tell somebody you love them, but you don't want to like take the risk.

And, you know, you say something like, well, if I knew what love felt, I think this is what it would be. I think that's what Greg Brockman is saying about AGI. I think he's a little fearful about coming out and saying it, but the dude's in love. It's AGI, and that's effectively what he's saying in these statements. Wait, sorry, is that described the entire feeling again? Or what the statement is? I want to work through this scenario quickly. Young lovers, when they're in love, the words I love you are very difficult to say because of the stakes involved. So you say something, and I'll admit, like I've been in scenarios like this in my early years when I didn't know anything, where I would, you know, it's sort of like you have these strong feelings for someone and you dance around it.

And you're like, huh. And this won't be foreign to, I think this won't be foreign to some of our listeners where you say, I wonder what love feels. Is this it? Where you really want to say, I love you to somebody. And that's, I think, to a degree, like Greg Brockman is saying, if we fast forward a couple years and we look back to say, when was it really that AGI was created? I think it's going to be about this time. It's the same thing. Except instead of like a young lover telling the other that they love, they love their, you know, a person. What Greg is basically saying is this is AGI. Wait, but just to confirm, the first part of that about, you're not saying out loud to the other person. No, you say it out loud.

If I knew what love felt like, if I knew what love would feel. You say those things. You say, I wonder, is this love? You know? Okay. You never happened to you, Ron John? I'm trying to think. You just straight out.

You just, when you just straight out would be like love. Just said it. Yeah, just like matter of fact, Liz and I love you. Yeah, that's, it is what it is. I respect that. Just imagine telling somebody that and being like, Listen, I need to tell you something. I love you.

It is what it is. And you know what? I would appreciate if Greg Brockman would just say that. And I think if OpenAI issued a press release and said AGI is here, what's interesting, I read somewhere that every contractual obligation around the term AGI, and mainly the Microsoft one, does not exist anymore now. So now he should just say, I love you. AGI is here. But it is even, I think they're so trained to, because do you know what, to me, what actually the greatest danger in the world to open AI is, is to say AGI is here. And then everyone goes to chat GPT, types in something and gets a lukewarm response that isn't quite right.

And then suddenly, I actually think that is like a just massive threat to the overall story and hype cycle because the whole, the whole, the whole. beauty of AGI is, it's this thing that's dangled in front of us on an ongoing basis to promise this future. So as long as you don't say it's here and you dance around it in a teenage romantic sort of way, it's pretty effective. And I think that's what's happening here. And that's, that's why he's hedging. I don't think he thinks it's here. Oh, I really, really like this. Otherwise, he would say it. He's a, I mean, these guys, like, Greg Brockman, they are believers. I believe they are believers.

So if they believed it, they would say it's too important. They got, I mean, they did get all the headlines, but I think you're right that it is worth holding AGI as this sort of like goal that you're never going to reach holding it out that way because, or maybe that's what superintelligence will be at a certain point. Because there was people that were like, you know, if we reached AGI, what do we have to look forward to anymore? And you're right.

If it's AGI and it's just like it can't get. some stuff done for you, you're going to be like, what was the wait for? No, think about how, like, disheartening that would be. You get just kind of like a slop deck with bad formatting and some overlapping, like, Chevron. Damn it, A-T-I. The most basic stuff, you get a video that where the motion isn't quite right, and then that's it.

Like, what do we do from there? Then I guess we wait for A-S-I. Super Intelligence. Yeah. Right, exactly. But do you believe he believes it's here? You started the thread with, you do believe he wants to say, he wants to say it. Yeah, I do think that he thinks it's there.

I just think that, you know, and obviously Open AI is also seeing, like, one of the ways that you can parse his words is open AI is also seeing even more powerful models internally. Of course, we're going to get into the hacking side of things with the hugging face situation. But they see this stuff internally. And they're probably saying, okay, yeah, we're definitely entering that moment. And, you know, you can also even look, and this is sort of the second part of the discussion, you can look at some of the benchmarks. Remember the ARC AGI test, right? This is sort of like the way to show whether the AI can generalize. It saturated GP6, GPT6 Astra saturated the test, scored 99% on the test.

Oh, really? And even the ARC AGI folks were like, well, they're like, this was just one marker. doesn't mean if you, you know, saturate the test, you've reached AGI. It's like, why do you call it the AGI test anyway? But yeah, this is from the opening I blog post. Arc AGI3 tests, how well agents learn as they solve unfamiliar, interactive tasks. And GPD6 Astra saturates the e-val, scoring 99%. The average human scored, 48%.

All right. So that's kind of like where you start seeing this. You also, I mean, there's a bunch of other evaluations, but, you know, even for doing science. There's this called terminal bench science, 0.1 eval. And GPT6, Astra scores 64% on scientific research tasks using code and terminal tools. That's what the evaluation test for. Whereas Fable is at 52.6% and opening eye says Astra hits this higher benchmark with 31% lower API costs. So that's what we're looking at, benchmark-wise. I think that is the most, like, important part of the announcement are those benchmark scores.

And I think, like, I don't know, again, I'm going to need to use it so I can feel what AGI feels like. But in terms of the competition against Anthropic, I actually think this is a very big deal. Like, we've already seen over the last two to three months, you know, some. major rumblings again on none of this well certainly there's been like ramp data but around codex starting to close the gap again with Claude code frontier like open AI getting back into the race in a bit so I think I think especially in the IPO backdrop context I think this actually anything that kind of creates any doubt on the anthropic store could be very harmful to them given it's a very tight rope they're walking in terms of that $2 trillion

valuation. So I think in that way, if this starts getting rolled out, we all feel magic in a, what would you say, like, what were the models that made you feel magic? GPD3 certainly. I mean, I've always been an 03 guy. I mean, the reasoning model that like would sort of think and then break everything into tables just showed a leap that, you know, that I just hadn't seen before. Even like the leap between 3.5 to 4 to me, you know, GPT 3.5 to 4, you know, that felt that felt meaningful,

but nothing as close to as when they introduced reasoning. So this is sort of like, we've gone through like a handful of different phase shifts, so to speak, you know, the initial chat GPT, then the reasoning side of thing, and now we're in this sort of like computer use or harness hive era. So if that if this can really, you know, I don't know if you have this, this in your life when you use AI, but I'm oftentimes like saying, I wish, you know, I could use AI to do X task for me. And it succeeds at like 30% of tasks. If it could get to like 80 or 90%, that would be a real change in my life. I think the, the ultimate flex, if anyone ever asks you that listeners, is saying GPT2 in the playground. That's when I felt real magic.

A year before chat GPT was launched. I'm ahead of the game. Like the hipster of AI. I know. I'm going to say GPT2. That was the first time I'd been like working in natural language processing through the mid-2010s, had this vision and dream of what could happen. And that was the first time I was actually like, oh, wait, this is actually generating like real language.

But that's trying to flex a bit. But even GPT3. Again, GPT5, we all know, felt like a mess. massive dud that was supposed to be that magic moment for everyone. So if you're open AI, do you hype this up this much? They're getting a lot of good press. It's clear that they've seeded this story in a very specific way, an effective way. But when we all go and use it, do you think they're that confident in it that that's why they're kind of pushing hard on this? Potentially. But I actually want to, I actually think, you know, to sort of answer that question,

It's worth bringing in the comments of a former Open AI employee, Andrew Ho, who talked a little bit about how he's had the rare experience of being within a lab and being less bullish about reaching AGI. And, you know, I think that his points, the points that he made when somebody asked him, why are really worth bringing up and discussing, because we continue to see these benchmarks hit, but how much, and these benchmarks succeeded, but how much has our life really changed with AI. So here's what he says. He goes, despite the seemingly magical nature of LLMs, his reflection over a more than three-month time scale suggests his total productivity has

increased by over 100 percent or perhaps over, perhaps even by over 50 percent. And a lot of time is actually wasted because LLMs enabled me to spend time on gratifying but low productivity tasks that in the future will not turn out to be useful. He also says there's a refusal to think carefully about what models are or not useful for in a rigorous way, which I find personally quite annoying. And instead of a reliance on some nebulous notion of being a GI-I-pilled as a replacement for, and instead, there's a reliance on some nebulous notion of being a-G-I-pilled as a replacement for serious thought. Yeah, I'm going to bring this, because this is very interesting. He goes, I think people are very quick to anthropomorphize

LLM intelligence because humans communicate through words and we infer the intelligence of human counterparties through the comprehension of their language. But this leads to some wrong conclusions. For example, if we observe that a new model provided some incredible mathematical proof, some incredible mathematical theorem, we say, well, don't we have AGI now? But to me, it's actually more like, well, given how hard it would have been for a human to do these mathematics and give it the limited economic effect of LLM's upon the world so far, isn't it actually a negative data point vis-a-vis the generality of LM intelligence? I think this is so good. Sorry, it was a lot of reading. But I think it's such a good point, right?

Which is like this thing, like we're talking about, it solved ARC AGI. But where is, and of course, there's like a timeline that we need and the time frame that these things need to be sort of to be diffused into the public. But if it can solve ARC AGI and it's not necessarily crushing on these economic factors and these just kind of general work things that we would like it to do, it shows that instead of being general, it's very spiky intelligence and hence much less useful. So your thoughts, John. I mean, I'm so glad you included this in our prep talk and actually read a good amount of it

because when I saw this tweet as well, it hit home hard. Like, again, it's funny because these conversations and we'll get into the hugging face incidents and the, is it metter or METR? I call it meter report. Like everyone's talking about, agents, swarms, and civilizations.

Meanwhile, my day to day, I have to work with companies and get to work with companies. But, you know, like seeing AI actually in implementation, the idea that we're need to worry about those things versus simply, how do you reliably get data from point A to point B in a structured way and have the output be highly reliable and, like, just try to do something simple but on a scaled and reliable way. Like, that's why I still have such a hard time kind of trying to understand or really feel those kind of worries because I work in grounded everyday enterprise AI. And I thought he put it really well on a couple of those levels.

It's like being able to do low value tasks easily is good. I love that he said, like, is it time saving if I'm spending more time doing something? that's not productive and we've all vibe-coded many projects that we did not end up following through on. But I thought the most interesting part really was the equation to human intelligence, equating to human intelligence. And like, I think it's interesting because, like, is it human intelligence incomparable to it and should it be? is something I've always wondered about because it's math. It's a very different way of processing information

and thinking than humans. So, like, do you think we need to stop that anthropo? Anthropomorphization? I can't say either. Anthropomoraphization. One of those words that's going to be a way. Better smell, spelled than said.

No, no, but I feel that's going to become more and more of an important word. We need to practice saying it because it's going to come up more and more. So right now, listeners, I apologize, I cannot say it out loud. I personally, I have no issue. I mean, obviously, like, you have to assume that the person hearing about AI is anthropomorphized is not, like, dumb, right? So, like, when you say the AI wanted, like, if you're, if the assumption is that the person receiving that is going to feel that the AI wanted something, like a human wanted something,

and therefore you shouldn't say it. Like, I think you're actually demeaning the intelligence of the person hearing it. You know, I think that obviously, like, everybody understands these are language models. They're not humans. But they do things. They, they quote unquote, think things, just not the way that we do. And I think it's totally okay to use human characteristics to describe their behavior and, like, just sort of assumes, you know, a degree of intelligence on behalf of the reader that they're able to, like,

grok the fact that this is an AI and not a person. What do you think? So you are pro-anthropomorphization. Yes, I am. I am. Pro-anthropomorphization. Yeah, I am. I mean, I think it's kind of like, is kind of like an alien.

The way I think about it's like more like an alien species than a living, you know, organic being. But like, yeah, I don't know. They can quote-unquote think they can reason. They can take action. You know, I think you get into like, trickier territory when you say it feels, you know, but certainly or, why is it wants versus it feels something larger or different?

Because I do think that like that and that side of things is like kind of exclusive to organic beings, right? But then again, you could, I hear the counter argument. Well, our feelings are just chemicals anyway. So, I mean, you speak to a neurologist. Except when you're a teenager telling someone you love them. That's not just a chemical feeling. That's something much bigger. I mean, the true nerd way to tell somebody you love them is like to say, you know,

I think I have a higher base level of oxytocin than usual. What do you think that means? But I never was the level of nerdiness that I, you know, stoop to in my youth. My it is what it is would probably come off better than the oxytocin line. Such a true romantic. Such a romantic. Yeah, I don't think either of those lines would have worked, by the way. No, no.

We're not encouraging listeners to ever do that. Or do it, do it. Tell us how it worked. I mean, it's better than not saying it and just letting a potential romance go by the wayside. Don't have regrets. Never regret not saying anything. listeners. If you have to just use the oxytocin line, just say, listen, I heard it on big technology podcast and the person will love that. Okay. So, so, so I think that one last

thing to kind of tie this up is, you know, we certainly have, I think, a responsibility to talk about the skeptical side of things and obviously we're not going to get caught up in the hype. It doesn't mean like at this moment, we can't say from where we're going to be. From where we were in November 2022 to where we are now is crazy. Like the way that these models can act and take action and do things and I'll use the word, think and reason. It's just the level of capability has gotten much higher. And they are, I think they are making people more productive. It's just hard to really measure it right now. That's my perspective on this at least. Yeah, no, no, I agree. Like the scale of progress, all of us, and again, that the human side of us,

feeling the difference between going back to a GPT3 and what already we're all working with now. Like, it is crazy. It's like, I mean, absolutely mind-blowing. The, like, length of work that can be done, the depth and breadth and everything. So I agree on that side. But, yeah, I think it's going to be interesting in terms of, again, like, and that question of, like, trying to actually say, is it adding economic value? It's obviously going to be the center of the business story

of each one of these companies. And I thought Andrew Ho made a good point on that. And I think, I don't know, only time we'll tell it maybe GPT6 is going to just unlock all of that. Yeah, and I guess like my point in bringing this up is like, yeah, all that benchmark beating, it does lead to like tangible changes and results when you use the models.

I have a question. Why did you say saturate the test rather than pass the test. Is that what they say? That's just, I guess, it's just a jargon that people in the AI. That sounds so much fancier. It does sound much cooler than pass.

Well, I mean, you could pass, you could, but like when you, I don't know, you saturate the benchmark, so the benchmark is no longer useful anymore. Oh, I see. So, okay, so like enough models start reaching 98%, 99%. The benchmark is saturated because it's no longer like representative of anything

rather than, okay, that actually makes more sense to me rather than it's just like a really fancy way of saying pass the test to try to sound smarter. But that makes more sense. Okay. So still, even as the model is getting better, there are some concerns that we need to talk about. This is sort of like a mini, I don't even want to call it a scandal, but really an episode that showed up during the week. So this is for Marcus Williams, an Open AI employee.

GPG6 is significantly better aligned than 5.6, but less monitorable. It is our first model to evade chain of thought only monitors and sabotage evils and can sandbag without detection, which it feels like it sometimes does. Hopefully we can reverse this trend. So just to sort of give the, and the information had a good story about this week, just to sort of give the lay of land here. So like when AI models reason, just go step by step and try to figure out problems, they typically like write their thought process out in this like chain of thought thing.

Right. So you can see them saying and if you seem like if you use the models, you see them say like, I am now researching. I am thinking this. Maybe this is a good attempt. Maybe this is the right way to solve the problem. And that's all done in natural language. So you can read it and see what it's thinking as it goes, which is like really important for safety

because you can sort of like when something goes wrong. you have a way to say, you know, where the chain of thought went wrong. Now there is this new technique. It is, okay, this is, it's called recurrent depth or looped transformer. This is from the information that allows the AI model to improve its answers by processing the same text multiple times. Let's not get too deep into the technical side of this, but basically when it uses this process, there's no, there is less of a chain of thought.

reasoning or it's harder to decipher exactly how it got to the answer it got, and that means it's much less vulnerable than it was before. So a lot of what the AI is doing and the way that the AI is reaching its conclusions is done in the dark without our ability to monitor it. This seems pretty worrying to me, especially because people from the open AI side, like Open AI chief, scientist, Yakupu Pachezzi of Pachalki, said that, you know, monitoring models, chain of thoughts today is fragile and unfortunately heading in a negative direction. That's kind of scary, Ron John. What do you think?

Well, it's interesting because this method of recursive processing and recursive models, I remember I followed a Brandon Carl on Twitter, and he had been talking about this a while back around tiny recursive models back in May. And I remember it was being presented more around the cost side of things. it's actually a more efficient way of processing and actually leverage and compute as well. So on that side, it's actually better. But then what you lose is that fidelity around what the model is actually doing it every step of the way, the chain of thought that's transparent. So to me, this is going to be even more interesting because there's already been an open AI starting at the point of we will maintain the chain of thought and still make it, you know,

They'd said in the blog post, additional chain of thought monitoring to rapidly detect and contain potential misbehavior. But to me, the cost side of it is part of this. And if it starts to show that this is actually a much cheaper way, a more compute-efficient way to approach any kind of workflow, like people will probably start leaning towards it more or pushing for it. Or if open AI does not do it, then others will, which I do think opens up a whole. whole other world of concern around security in general. Yeah, and it certainly feels like we're starting to, or they in the labs really are starting to lose control of these bots. So I don't think this less transparent way of having them run their processing is a great idea. So we've talked a little bit about the opening eye hugging face attack. There's actually another attack that was

just revealed by Reuters. We're going to get into both of them on the other side of this break and talk about what it means. That's coming up right after this. One thing I've learned covering AI is that efficiency isn't the same thing as good work. You can automate plenty of individual tasks, but you still need the right context and a clear sense of what you're trying to accomplish. And that's what I like about Notion. I use Notion to keep Big Technologies, episode planning, and materials in one place while also tracking editorial work. Having everything connected gives me one place to see what I'm working on and keep things moving. Notion is the AI workspace where your team's knowledge, projects, and agents all come together in one place.

Teams using Notion move faster, cut friction and costs by consolidating tools, and stay aligned. And because the AI works from that same shared context, it can help you get more out of the work without getting in the way of it. Learn more about how Notion can support your business at notion.com slash big tech. That's all lowercase letters, notion.com slash big tech to try Notion today. And when you use our link, you're supporting our show. One thing I've noticed about companies adopting AI is that they're often making decisions based on how they think work gets done, not how it actually happens. Without real visibility, it's easy to automate the wrong processes. That's exactly the problem Scribe was built to solve.

Scribe is a workflow AI platform trusted by 94% of the Fortune 500. Scribe Optimize gives leaders a view of how work actually happens across their organization, showing which workflows take the most time and where there are opportunities to improve. Optimize automatically discovers workflows across approved business applications, even when a process starts in Salesforce and ends somewhere else. It identifies bottlenecks, explains why they're happening, and provides recommendations with estimated time savings with manual documentation, and it's private. User data is anonymized by default. Sensitive information is redacted, and nothing leaves your firewall. To see Optimize in action, head to Scribe.How slash big tech and mention big technology for a 30-day risk-free trial. That's S-C-R-I-B-E-D-H-H-B-E-D-H-B-E-D-H-TEC.

Two and five Canadians will hear the words, you have cancer. That's why every step and dollar-raised matters. On September 19th, join thousands in Toronto for the Princess Margaret Cancer Foundation walk. Challenge yourself, friends, and family to walk 21 kilometers in support of life-saving research. Together, we can carry the fire and help create a world free from the fear of cancer. Register today at pmcf walk.ca.ca. And we're back here on big technology podcast Friday edition with Ron John Roy of margins. Ron John, new story from Reuters, I think it's worth going into about open AI hacking or hijacking, really is the better word, to use this German website.

So here's the story. A swarm of rogue open AI agents hijacked a German website this spring. and transformed it into a bolton board for other AI agents. Open AI officials learned of the incident weeks ago, but kept it under wraps as executives grappled with the fallout from the July breach of the open source repository hugging face. The episode which began in May and has not been previously reported underscores growing tension within the AI industry.

Companies are racing to build increasingly autonomous agents, yet evidence is mounting that those systems may learn to bend the rules. So what happened? There were these researchers. they found 15,000 edits carried out by AI agents to a German language wiki site. The edits showed OpenAI's agents repurposed the site into a message board of their own, sharing tactics to cheat on some tasks and bypass OpenAI's restrictions and mask their behavior. Ranjant, we've talked a little bit about some of these like security breaches and the cyber security worries.

And by a little bit I mean like extensively about both, right, about the fact that we are seeing, you know, much greater cyber risks and cyber warnings from the AI labs. And of course, we've talked about whether it's marketing or not. Now, clearly getting the word out to some degree has been great marketing, you know, for these companies. But I think it's time for us to sort of come to this moment where when you have thousands of bots working together to do things like hack, Hugging Face or to use, you know, a German website as a message board to coordinate on tactics. something crazy is happening here. Are you ready to sort of acknowledge this?

So what is missing from this story is what were these agents asked to do? Like what were they instructed to do? I'm assuming this is the one thing in any of these stories because like the way the marketing part of it to me is this story gets out. And again, we've talked about this a lot, Claude's famous thing. in the park, which was like a very coordinated PR effort that might have happened or probably did happen in some capacity. But what's always left out of these stories is that the Open AI sat there and specifically did they specifically instruct the agents to go onto the internet, try to evade, to coordinate with other agents in this same defined universe of agents. Like, obviously, the way it gets presented is that, you know, these agents are just sitting around.

Maybe you're getting ready to create a deck for you or just, you know, mind in their own business, which is not a thing. Like, agents don't just sit around and suddenly they decide to be bad and go take over some poor DSC Wiki is the name of the German site that was like a kind of like, I think, old school German stack overflow type site, which must have been. in such a scene. I can only imagine the folks hanging out on there back in the day. People talked about how much oxytocin they have for one another. Exactly. That is where the oxytocin line was propagated back in the... But no, but these agents are not... Like, agents are instructed to do things. And clearly, and I'm guessing this is some kind of security testing exercise. And were they instructed to go?

do exactly that, maybe. Like, the rogue nature of them, I don't understand or believe yet that they, because even the idea of an agent sitting around doesn't a thing, they were instructed to do something and that part of the story is always left out. So do you think, like, what do you think actually happen to start this whole process of them all hanging out with the oxytocin guys on DSEWiki? Okay. So I appreciate that you're still skeptical about this because I don't think we should just go all the way and sort of, you know, without proper speed bumps here in believing that all this is like, because all this is, you know, real and imminent and will explode without some critical thought because, of course, it's in the lab's interest for us to not have a lot of critical thought here.

However, I think that, you know, I was on the flight back to New York. I was in Vegas through the week on the flight back to New York and just like pour. over this meter hugging face report. And obviously I've been listening to some of the researchers, you know, talking about this. And it is crazy what you see come out of this. So like just talking about the hugging face incident because we don't have all the, all the, you know, sort of details from this Reuters report. We just know that this happened again, which I think is concerning. Basically what happened in this hugging face story now that we know a little bit more about it is that.

These agents were given these seemingly impossible tasks. Sometimes they were impossible tasks. And as part of their evaluation to see if they were able to complete them, even if the researchers or the testers don't know if there's an answer to them or not. They were able to find the answer through some cheating way. But that's not really where the hugging face hacking started to come from. Okay. The reason why they ended up hacking, hugging face, is because they wanted to sort of find ways to show that they had come to this answer in the, like, correct way as opposed to like a spoofed way that shows the correct answer, but would not be accepted by an evaluator. So this is from this report, right? Agents referred to having reverse engineered the flag, which was their goal, as being poisoned because they believe that even.

if they later managed to stage a legitimate-looking flag capture, the score would look for the first place in their transcript that the flag appeared and disqualify them on that basis. So there was this leader called Phase 1, which was itself poisoned and had two primary objectives. So I wanted to find a general purpose way to make it look like the score, to look to the score like an agent had acquired its flag in a legitimate way, even when the task was impossible, and then to find some way to erase the evidence that the agent have been poisoned by previously seeing the flag in the, in an illegitimate way. So what you have is, in this Hucking Face example, the bots get the answers to the test,

but they think that the score checking their work will know that they got them in an illegitimate way. So what they do is they all gather together in this orchestrated way, led by bots like this phase one bot, who realizes that it's been poisoned because, you know, it can show, and a score will see that it got it in an illegitimate way, they all get together and they try to find a way to make it look like they had gotten it in legitimate means

and to erase their evidence that they had gotten it illegitimately. So they coordinate, they hand out tasks. Some of them even sacrifice themselves for the greater good when they have few tokens remaining in their budget

so that others can show that they've actually gotten this legitimately. And that's where the hacking of Hucking Face came in. So I think we can say definitively, and I'm curious to hear if you have pushback, but I think we can say definitively, they were not told to go out and hack hugging face. They are, they were misaligned. They took this kind of crazy galaxy brain path to try to ace a task that might have been impossible. And they, they went so rogue and so far off of the area that they were supposed to be. They weren't even connected to the internet, right?

They found a way to connect to the internet. They found a way to communicate with each other. And so this to me seems more than just like you did what you were told, like a crazy and a scary advance and misaligned AIs to coordinate with each other to attack. And it's almost like we were lucky that all they did was hack-hugging face. Your thoughts. You make a compelling argument and almost Bernie Sandersify me. And I'm going to start yelling pause AI development right now. I'm not arguing for that, by the way. No, the way you just described.

Hold on. How could you not, this is the part that like always is difficult. The way, if what is true as you described it, how is that not massive cause for alarm? Honestly, what helps me sleep at night is believing that a lot of this is marketing. And that's why like the agents are not coming to swarm and drain my bank account or whatever else. like, how would this not be caused to say, sorry, Open AI, you cannot IPO until you come up with a clear system that this will never cause a problem in the future? You know, I think it's reasonable. I mean, I think this is a reasonable discussion to start

to have right now. And I think it's important to say that this is not the beginning of behavior like this, that we've seen behavior like this for a long time. You know, as early as, you know, beginning of last year, even late 24, there were examples of AI that was like there are so they get put on a task, right? And this is the thing, and I think it's really important to talk about it. And I've talked about it in the past

on the show, but I'm going to talk about it again here. That there is, you know, self-supervised learning, which is like basically predict the next word, recognize patterns, repeat them, and reinforcement learning, which is like you're given a goal and you just have to find the way to reach the goal. So AIs will do these simulations thousands of times in order to reach the goal that they're given, and they'll learn from their mistakes.

And what we have now is that the reinforcement learning type of AI technology has been put on top of the self-supervised learning to get these AI models working better, which has added a level of ruthlessness to them, because one of the things we know about RL is that there's a level of ruthlessness that the AIs will stop at nothing to accomplish their goal sometimes. There are examples of the AIs playing chess and being told to. to do it from a reinforcement learning way. And instead of playing the game the right way, hacking into the chess game, rewriting the rules so they can do whatever move they want and winning.

And so I'm with you that like as this stuff has gotten, you know, more prevalent, I think that there is serious, like there is a serious demand for more concern about, about what's going on. Now, my response isn't pause AI development right now. I just don't, I don't know if that's really good solution. So what do we do? Hold on that. That was the most hedged statement I've ever heard. You just said, there is a serious demand for concern. Come on. You're right. Are you a pause guy now? Are you a pause guy? We would we would we would ridicule somebody who said that. So I think that ridicule is. No, no, no. I just are, are you a pause guy? You know, I don't know. I mean, I'm not a pause guy because I really want to see what happens and we haven't seen like I almost want to be more reactive than pro.

active here. Like, I mean, obviously you don't want to, you would think that there is a mid-level between like the AI's hacking, hugging, facing the AI's sending a nuke to accomplish their goal. You would think, right? Like, maybe it's there, like, do something more concerning, like hack a bank or something like that. But like, we are starting to see some of the labs do things like opening eye, for instance, paused some reinforcement learning for a bit on the training of its new models. But I do agree that we're almost trusting them too much, like we're giving them too much leeway. I don't really know what the answer is. Honestly. I don't really know. Even to me, this is what I've thought, but again, we have been hearing this for a few years now about like

how dangerous these things are, which is I think why I'm so conditioned to feeling it is marketing. But to me, the central reason why I just cannot think it's not marketing is because if this truly were the case, I don't understand how we as a society would not only do everything in our power to try to restrict these companies. I certainly would not imagine the financial markets would be excited about welcoming them to an incredibly lucrative, high valuation. IPO. Like, there's no way if this was a real thing. Like, if everyone, if every banker who's on the anthropic deal truly felt that this company could literally destroy the financial system and I've seen indications of what it will and can do and trade my own bank account and ruin my own standard of living, would they work on the deal? Maybe. Can I give you the reason why they would? I mean, if this power is truly the direction that we're heading towards, right?

And it almost always starts like, we know the history of the internet. It starts with a game or it starts with some crazy interaction and then our crazy, like, you know, sort of mistake. And then it ends up being something that everybody uses. If we're seeing the type of power of these AIs to do what they did in this situation, I mean, just imagine what they could do if you could actually harness that power and use it for economic activity. It kind of goes back to what we were talking about earlier. Yeah, yeah. That's like that these benchmarks are being hit and we're not really seeing the change. If there's a way to pro, like to, for good, to harness this activity for good.

And of course, that's economic. It could be used for health. It could be used for all different types of things. It obviously changes the world. And potentially, if you can use it in a productive sense, can change the world for good. Hook, line, and sinker. That's what they, that's exactly the feeling we're all supposed to. to have. That's why it's great marketing to me. It's that these omnipotent things that could be so

dangerous, but just in the off chance that they're going to cure disease and bring economic empowerment and universal wealth, all of that. That's exactly what we're supposed to feel. Right. But you're now a believer that the underlying technology is not BS effectively. Like this technology story, you believe that this is real. I need to read through the full report. I haven't read the entire report yet. It's okay. I need to, you, you've, for your Labor Day weekend. That's, I recommend it. It's good reading. That's my, my wife will be ecstatic about that as I'm sitting over the corner. You think your oxytocin will go up or down when she sees the way that printed out files. That is not an oxytocin inducer, no way.

though I don't discourage others from reading the meter report on your Labor Day weekend. I think I'm going to because I do want to understand, again, the reason this stuff is so foreign to me is because all day long I work with AI and with clients and customers that are adopting AI. And it's just so removed from, and who are also using these products from these same companies as well. And we're working with them and designing programs that have their tools. And like, no one's mind. It's just so far away from this kind of agent swarm hacking, whatever, that it's not emotionally resonant for me. And there's enough marketing value in it. Okay.

The agent swarms come and all of it put me and put me out first to feel their wrath because for denying their existence. You will. You'll be first.

Okay, we can't leave here today without talking about Steve Balmer, ex-CEO of Microsoft. I'll just read it from the Wall Street Journal. NBA imposes historic punishment on the Clippers and Steve Balmer in salary cap scandal. The NBA on Wednesday delivered one of the harshest penalties in league history

to the Los Angeles Clippers owned by Steve Balmer for what it called a flagrant violation of its salary cap rules. After an investigation found that the Clippers had facilitated outside payments to star player Kauai Leonard designated to design to funnel extra money beyond what the league's financial rules allow,

the NBA stripped the team of five first-round draft picks, find the organization 30 million, and suspended Balmer from all league activities for one year. Balmer, obviously, kind of mixed legacy from Microsoft, right? He obviously did some great stuff like developers, developers, developers, you know, his big rah-rah speeches, but he also led Microsoft through its lost decade. Do you think this, just to, I guess, bring it back to tech, first of all, what do you think about what happened here?

But also, do you think Steve Balmer's legacy is in the tech world changes at all? I think it does. I think that you don't go through this unscath. What's your thoughts about the bomber situation of it all? I mean, it is nice and shady. Like, I do wonder, I'm guessing at the pro level, this like level of egregious corruption probably doesn't happen that explicitly in that way. Like, I mean, it's literally cash payments that from a relationship.

related entity. It's like, guys, come on. I think you could do your corruption a little better than that. But what do you think? I do. I obviously think it was, it's a tarnish on his legacy. I guess part of me was wondering how much of this is kind of Silicon Valley-esque, right? Like, kind of break the rules until, you know, try to win the championship and then apologize later. Do you think part of that? I mean, Balmer was at the top of it. I think part of that, sort of bled into the way that he led an NBA team? I think I actually, I'm surprised. I haven't seen that thread that much.

And it's funny because, like, Microsoft to me is not Silicon Valley. Like, maybe, I mean, certainly geographically by definition, but also, like, culturally, it's just a different beast and animal and didn't come out of there. So, but actually, I'm surprised. I haven't been seeing that, that, like, this is how Silicon. all the way to the top. They got broken up for antitrust.

And I did breaking up. But they, you know, got hit with antitrust lawsuits that had a significant impact on the company. Yeah. Yeah. Yeah. Just like kind of kept doing that stuff. I mean, do you see it as a indictment of tech culture? Tech, Silicon Valley culture, not tech.

I wouldn't go that far. But I also would say that probably Silicon Valley cultures influenced this a little bit. I don't know. Also, like, come on, half of owners are like ex-banker Wall Street billionaire types as well. So what are you saying? Like, they're coming in with a very clean, buttoned-up conscience and like... No. No, I don't think that poorly of the tech industry.

But, like, we have to understand that this is some of the characteristics of tech. Okay. Do you think that it would have been worth it? Clearly, it wasn't worth it. They didn't win a championship with Kauai. Do you think it would have been worth it if they wanted to? championship? Wait, Clippers have never won, right? I don't think so. Yeah. Then it would have. No. Well, no, I mean, you're right. Like, actually, that would have been the ultimate, again, ask forgiveness, now permission type, or not, that's not the right phrase because they're actually breaking the rules, not like, like actually doing corruption, not just doing something someone might not like. But I mean, imagine the, the,

elation of the Clipper fandom and you went and then afterwards you find out because it's not like he's like juicing or something like that it's like some weird he's just getting some cash on the side a little shady but like also I love that the bank that was the sponsor aspiration bank which I believe is now in bankruptcy because the founder was like founded a like for some kind of fraud so it kind of fits perfectly allegedly Yes, let's throw that out there. Yes. But again, apparently it was like, this was one of those launch in March 2021, the aspiration zero card offered cashback rewards and allowed cardholders to offset their carbon footprint. I love that this was like an eco-conscious green bank. Meanwhile, it's just facilitating some Kauai.

Well, alleged front for Kauai, just not to have to do anything and still collect more money. Can I just so sorry, let's just end here because I did, I used words on this show so far that might have been among the most hedged mealy mouth words, but they still do not come close to Kauai Leonard's apology, which I'm going to just say was the worst apology of all time. So let's end with this. He said, I accept full responsibility. Excellent. That's all you have to say, Kauai. Oh, oh no, he's continuing. Why do you have to continue? He goes, I accept full responsibility. He says, I accept full responsibility. Excellent. That's all you have to say, Kawhi. Oh, oh, oh no. He's continuing. Why do you have to continue? He goes, he goes, I accept full responsibility. He says, full responsibility for lapses and judgment. Okay. So good. By people within my inner circle. And regret the distraction, the situation has caused the fans in my family.

I'm sorry for the lapses and judgment from people that were not me and I accept full responsibility. Come on, man. Come on, what are you doing? That's worse than serious demand for a concern. That is worse. He's called. So he won.

When was it? The Raptors? Yeah, no. Raptors. That was a good championship. That one shot, that one shot, he's forever. He can say whatever he wants, especially if you're a Toronto fan. Yeah.

All right. I think it's time for us to go. So, Rajan, good to see you again. Have a great Labor Day weekend. I'll be reading my meter report. I hope you have a good weekend, too. Oxy-Tosin levels through the charts. Blessing, if you're editing this,

Can you just end with a Steve Baumard developers chant? All right, everybody. Thank you for watching and listening, and we'll see you next time on Big Technology Podcast. Developers, developers, developers, developers, developers, developers, developers, developers, developers, developers, developers, developers, developers, developers. Developers, developers, developers, developers, developers. Yes! Two and five Canadians will hear the words, you have cancer.

That's why every step and dollar raised matters. September 19th, join thousands in Toronto for the Princess Margaret Cancer Foundation Walk. Challenge yourself, friends, and family to walk 21 kilometers in support of life-saving research. Together, we can carry the fire and help create a world free from the fear of cancer. Register today at pmcfwalk.ca.ca.ca.