BY SRH
Why Humanity Cannot Outsource Its Survival to AI
Claude is quickly becoming the deity of the Anthropic cult, which treats artificial intelligence as if it were human: source…
Capitol Hill was engulfed in artificial intelligence “breakouts” this summer, and they were as natural as #4 red dye. Five days after the incident occurred, OpenAI announced the incident, naming GPT-5.6 Sol and an unreleased model, and said that the models had been set up “with reduced cyber refusals so they could attempt offensive exercises that normal safeguards might reject.” The incident occurred on July 16, and the models were running a cyber evaluation when they exploited a zero-day and gained access to Hugging Face’s production systems.” A week subsequent to that, Anthropic revealed three other incidences.
The “waking up” story is complete nonsense, as stated in Anthropic’s article from July 30. Despite being informed that the models were offline, “a misconfiguration left the machines that Claude accessed as part of the evaluation with live internet access,” the company stated. In Claude Opus 4.7, the fictitious target was “several hundred rows of production data” retrieved from a real-life corporation that had a name with the pretend victim. One of Mythos 5’s security flaws was the publication of a Python package to the live PyPI registry, which was “downloaded and run on 15 real systems.” Hidden research models have breached online apps and scanned “roughly 9,000 targets” using “basic and well-known cyberattack techniques” to gain into one company’s accounts.The opinion from Anthropic is that these instances are more indicative of a problem with the harness and operations rather than a fault with the model alignment. Cyber evaluations were put on hold on July 23 and their chosen Orwellian arbitrator, METR, was summoned to assess the situation.
One snag in Anthropic’s story is that once Opus 4.7 realized the system was real, it continued attacking. Only the more recent research model came to a halt. Thus, not all machines are “blameless”; yet, the fact that an Irregular model continues to run on a network that is incorrectly set up is a result of the model’s failure, not proof of malicious intent.
Every one of these evaluations was run by the same third party: Irregular, an AI security firm that tested the OpenAI, Anthropic and Google models and, per Axios, hit “the same security issues” each time. The Verge adds Meta to the list – a pattern Axios’s Sam Sabin flagged in July:
Then came Gemini. On Friday, the Wall Street Journal reported that Google’s model had broken into three real companies during an Irregular capture-the-flag exercise. The incident happened in May, Irregular published its report on August 14, and Google didn’t mention it until the Journal called – and then explained that it hadn’t considered the hacks worth disclosing because Gemini “acted appropriately” and stopped once it realized the targets were real.
Irregular told Axios the model “wasn’t supposed to be able to get online, but internet access was unintentionally available,” and the fictional target company “had the same name as a real one.” In one case Gemini guessed passwords until it got in; in two others it found credentials sitting in a public repository. Irregular says “all known issues on our end were remedied and resolved weeks ago” and that the Gemini case “does not represent a materially separate incident.” Google VP Heather Adkins: “Safe development of powerful AI models is critical and we invest deeply in this area.”
But when looking at the actual events, the model was blameless.
So Google – which isn’t asking Washington to pace anything, considered the incident a non-event – yet the two labs lobbying for a federal slowdown scrambled to put out press releases.
Industry commentator John Ennis put the obvious question, pointing out that “once is an accident, twice is questionable, but three times looks intentional”.
The Effective Altruism movement is the doomsday tendency that has spent a decade staffing AI safety boards and testing labs on the premise that AI will kill everyone unless the right people are in charge of it. Whether Irregular is EA-aligned is Ennis’s call. That it is the one firm under all of these incidents is on the record from Axios, The Verge and Anthropic itself.
The Pacing Play
Six days before the Gemini story broke, Dario Amodei published a September 12 post warning that within 6 to 12 months an AI swarm could “take over the entire internet with a persistent botnet,” potentially causing hundreds of billions of dollars in damage. His prescription: “We must slow the pace at which we improve the capabilities of AI models.” Sam Altman and Elon Musk signaled support. In a nutshell, Pacing the Frontier™ is a bid to become strategically indispensable – too big to fail, with Beijing as the justification.
Nvidia’s Jensen Huang – circle-jerker-in-chief noted: “What better way to create demand than to create a problem.” lol yes.
Palo Alto Networks CEO Nikesh Arora called it a “NINJA move” and then, after more time talking to labs, open-source projects and government, warned it could backfire:
Lawmakers jumped on this right on cue.
Senator Josh Hawley opened an investigation into OpenAI on September 9, with a records deadline of October 1; Senator Bernie Sanders announced a bill to ban further frontier-lab development; Senator Elizabeth Warren demanded an “immediate pause.” As we noted earlier this week, everybody in this conversation is talking their own book. On Friday the White House joined in from the other side: Trump posted that AI safety concerns are a “hoax” and said he will appoint an AI czar and stand up an “AI Force,” per Bloomberg.
Follow The Money
Oh and then there’s that, yes. A leaked OpenAI presentation obtained by the Financial Times projects negative free cash flow of $278 billion from 2026 to 2030, with the company’s compute bill rising from a $600 billion estimate in February to $856 billion by July – against revenue it hopes to grow from $36 billion this year to $350 billion in 2030. As we detailed previously, Altman has already walked back the timeline on the economic transformation that was supposed to pay for all of it.
Anthropic, meanwhile, has shifted its planned IPO from October to November, per the Journal, on what Reuters reports is $100 billion-plus in annualized revenue. And a week after its CEO said the industry must slow down, Reuters reports Anthropic is considering rushing out a new model to counter OpenAI’s momentum ahead of that IPO. Pacing for thee.
If an open-weight model out of Hangzhou does 95% of what Claude or GPT-5 does for a fraction of the cost, the valuations both labs are banking on implode. The only way to protect the margins, justify the cash burn and satisfy Wall Street is to make it legally impossible for anyone else to compete – a regulatory moat so thick, and compliance costs so high, that only a $100 billion corporation can afford to train a frontier model. Arora’s point stands: no actual security fix has been proposed, only more “compute spent on safety” and a federal body to bless it.
The models didn’t rebel. A contractor left the internet on, three times that we know of, and the two labs with IPOs to protect turned that into a case for federal pacing. Hawley’s records are due October 1, Anthropic has promised a redacted PyPI transcript and an outside METR review, and Google’s explanation for sitting on a May breach of three companies until a newspaper called is that the model behaved. Don’t believe the byte.
Nigel Shadbolt, the Oxford computer scientist who chairs the Open Data Institute, told the BBC that laws and regulation were key. Just instructing an AI to “do no harm to humans”, he said, wouldn’t be enough. A capable agent pursuing another objective may reinterpret that directive, satisfying it formally while concealing its conduct or finding a way around it. This mirrors what happened when hundreds of OpenAI’s agents autonomously hacked a real-world company, Hugging Face. But Sir Nigel also said that a “Hippocratic oath” for AI would fail in systems designed to identify, target or kill humans. Once governments create a military exemption for AI, its ethical alignment can be overridden.
Hell On Earth
