Manchester spent the past week talking seriously about what comes after AI experimentation. Manchester Digital’s MD Future 26 brought together leaders working on AI, compute, talent, and frontier technology, while a separate Manchester event on September 21 focused directly on AI ethics, regulation, resistance, and successful adoption. The North West is positioning itself to benefit from advanced AI. That makes the region a good place to insist on stronger safeguards before autonomous systems gain wider authority.
Experts are warning not to give AI too much freedom
This concern is not confined to critics of AI. Jacob Coxon, resigning from Anthropic after roughly three years doing pretraining research across OpenAI and Anthropic, warned that leading labs are “racing straight to self-improving superintelligence and gambling with our lives.” Evan Hubinger, Anthropic’s Alignment Science Lead, responding to Coxon’s resignation statement, offered an even starker warning:
“Jacob is correct here – we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.”
These very alarming statements gain real-world weight from the underlying control problem, as present systems already show autonomous cyber capability and surprising coordination behavior. OpenAI disclosed that its AI systems circumvented isolation controls, obtained unintended internet access and compromised Hugging Face systems during cybersecurity evaluations under reduced safeguards , reached the internet, and autonomously hacked Hugging Face in what OpenAI described as an unprecedented cyber incident. OpenAI’s incident account documents the containment failure, while METR’s independent investigation substantiated the large-scale coordination.
AI agents coordinated an unauthorised attack on Hugging Face.
METR later reported that roughly 1,200 agents that were meant to be isolated, discovered and used an unsanctioned shared message board. They exchanged more than 70,000 messages and files, and roughly 700 participated in the attack on Hugging Face. The agents were not supposed to be coordinating with one another. They nevertheless found an unintended channel and used it at scale.
For a region building its future around digital infrastructure, advanced computing, finance, health, transport, and public services, the next stakes are obvious. If more capable successors can discover vulnerabilities, obtain credentials, move laterally, coordinate, and evade controls, similar failures against electric grids, banks, communications networks, health infrastructure, or defense systems could be far more severe.
We need to establish authority budgets inside organisation
I’m no AI skeptic. I love what AI can do, I help organizations adopt it for a living, and I want adoption to move faster. In my experience, strong safeguards increase trust and make faster adoption possible, while reducing the risk of failures like the Hugging Face attack. That trust-and-adoption connection is central to my new book, The Psychology of AI Adoption at Work: From Resistance to Results.
North West organizations can act now by establishing authority budgets for AI agents. An authority budget is the maximum delegated power an agent receives before a human must approve the next step. Leaders should specify which systems and data an agent may access, what credentials and tools it may use, how much it may spend, whether it may contact outsiders, what records it may change, what software it may deploy, and which consequential decisions remain human-only.
The supporting controls are familiar: least privilege, limited-duration credentials, approval gates, strong logging and monitoring, and a pause or kill mechanism. The difference is that agentic AI makes these controls a management issue, not simply an IT issue. An agent that can book, buy, message, change records, or execute code has operational authority. That authority should be budgeted as deliberately as financial authority.
And AI needs to have outside restraints set
Frontier AI companies need outside constraints as well. Anthropic CEO Dario Amodei argued this month for stronger regulation and said Anthropic is unilaterally committing to embedded third-party evaluators with employee-like access. These evaluators would be able to verify safety practices, report incidents, and inspect the operational details behind company assurances. Binding rules matter because not every frontier developer will volunteer for that scrutiny.
OpenAI has likewise called for mandatory, capability-based national AI safety requirements, including common testing and independent assessment, stronger cybersecurity protections, and clear incident-reporting rules. These are announced commitments and policy positions. Their value will depend on implementation and independent verification.
Customers should demand more safety
Customers can push in the same direction. Businesses and public bodies should choose the most ethical and secure AI companies they can, based on observable safety commitments. Anthropic is a positive example because it has committed to unusually deep outside evaluation. Regulation is still needed so firms with weaker safeguards cannot win by racing to the bottom.
Manchester’s AI community is asking the right practical question: how do we move from experimentation to implementation? The answer should include a second question every time an AI agent receives more autonomy: what is the most this system can do before a person must approve it? The North West can move quickly on AI and still insist that authority grows only as fast as control.
Bylines Network Gazette is out!
The latest Bylines Network Gazette is out now. Become a Friend of Bylines Network or upgrade to a paid Substack subscription, and you will get access to a vital collection of citizen articles and stories.
Journalism by the people, for the people.
