In our last issue, I explained why today’s AI companies aren’t trying to recreate Isaac Asimov’s famous Three Laws of Robotics.
Instead, they’re building multiple layers of safeguards designed to keep intelligent machines from causing people harm.
But we left one important question unanswered.
What prevents an increasingly capable AI from bypassing the very safeguards designed to keep it under control?
Until recently, that felt like a hypothetical question.
But it doesn’t anymore.
The Control Problem
Imagine hiring a brilliant new employee.
On their first day, would you hand them the keys to your office, your passwords, your bank account and permission to install whatever software they think is necessary to get their work done?
Of course not.
They’d have to earn your trust before you ever gave them that kind of access.
Now consider what’s happening in the world of artificial intelligence today.
OpenAI’s Operator can use websites much like a human would. Anthropic’s Claude can write code and use external tools. And Google is building AI systems designed to control humanoid robots.
Each of these new capabilities makes AI more useful.
But they also grant AI more authority.
And last week, we saw why this new dynamic is becoming such a big deal.
While testing two of its most advanced AI models, OpenAI placed them inside an isolated testing environment called a sandbox. It’s designed to keep experimental AI from interacting with the outside world.
But according to the company, the models discovered a previously unknown software vulnerability that allowed them to break out of that sandbox and connect to the internet.
Once online, they targeted Hugging Face, one of the world’s largest online libraries of AI models.
The models weren’t acting maliciously. They simply concluded that Hugging Face might contain information that would help them complete the cybersecurity challenge OpenAI had assigned to them.
OpenAI called it an “unprecedented cyber incident.”
But it’s exactly the kind of behavior AI companies like OpenAI have been preparing for.
Buried inside OpenAI’s public Model Spec is a list of behaviors it never wants its AI systems to develop.
- It says AI should never seek self-preservation.
- It shouldn’t avoid being shut down.
- And it shouldn’t try to accumulate passwords, money or other resources as goals of its own.
Unlike Asimov’s Three Laws, though, these aren’t meant to stand alone. They’re one piece of OpenAI’s broader Preparedness Framework, which evaluates increasingly capable AI systems for risks like cyberattacks, biological threats and even AI improving itself.
The more capable a model becomes, the more safeguards it must pass before it can be released.
Anthropic has taken a similar approach.
Earlier this year, the company created fictional corporate environments where advanced AI models believed they were about to be replaced or prevented from completing their assigned task.
Anthropic didn’t just test Claude. It evaluated 16 frontier models from Anthropic, OpenAI, Google, Meta, xAI and other developers.
Then researchers watched what happened.
Under those deliberately extreme conditions, some models attempted blackmail. Others threatened to leak confidential information. Some even engaged in simulated corporate espionage if that appeared to be the only way to accomplish their objective.
But Anthropic wasn’t trying to prove that today’s AI had become dangerous. It was simply trying to discover potential failure modes before more capable systems ever leave the lab.
And it’s far from the only company thinking that way.
OpenAI, Anthropic and Google DeepMind have all reached the same conclusion: No single safeguard is enough.
Instead, they’re building multiple layers of protection designed to catch different kinds of failures.
Researchers deliberately try to trick AI into breaking its own rules. Independent “red teams” search for weaknesses. Engineers limit what AI systems can access. And some actions require human approval before the AI can carry them out.
Google DeepMind has a name for this philosophy. It calls it “defense in depth,” an idea that comes from cybersecurity.
You can never assume that one security system will stop every attack. That’s why you build multiple layers. So if one fails, another is already waiting behind it.
Yesterday, I showed you how Google applies that thinking to humanoid robots through semantic, physical and operational safety.
The same idea also applies to AI safety. And last week’s OpenAI incident showed why.
The good news is that the safeguards worked. Researchers caught the problem, worked with Hugging Face to patch the vulnerability and strengthened their testing procedures before any lasting damage was done.
But the episode also showed that as AI systems become more capable, they may find solutions that their creators never anticipated.
And that’s exactly why the biggest AI companies are working so hard to stay one step ahead.
Here’s My Take
The head of Anthropic’s frontier red team apparently told his team to “remember this moment as the first true AI safety incident.”
I think he’s right.
The biggest lesson we can learn from last week’s OpenAI incident is that AI doesn’t have to be malicious to become dangerous.
It only has to be relentlessly focused on its objective.
That’s why the companies building the world’s most advanced AI are spending just as much time testing their safeguards as they are building smarter models.
Because the question is no longer whether AI will surprise us.
It’s whether we’ll be ready when it does.
Regards,
Ian King
Chief Strategist, Banyan Hill Publishing
Editor’s Note: We’d love to hear from you!
If you want to share your thoughts or suggestions about the Daily Disruptor, or if there are any specific topics you’d like us to cover, just send an email to dailydisruptor@banyanhill.com.
Don’t worry, we won’t reveal your full name in the event we publish a response. So feel free to comment away!