Models & Labs · United States

METR finds Anthropic may have crossed its own AI acceleration warning threshold back in July

Credited to The Decoder · the-decoder.com · 15h ago

Nonprofit METR got eleven weeks of access to Anthropic's internal research data. Its finding: the rate at which Anthropic's models are improving has roughly doubled, and about 80 percent of that acceleration traces back to AI being used in the company's own research. Anthropic redacted the most important evidence, though.

Whether AI is speeding up the development of better AI is considered a key early warning signal in the safety debate. The concern is that a feedback loop could form where models improve their successors faster and faster, which according to METR could lead to a "sudden emergence of extreme and risky capabilities." Anthropic flags exactly this kind of acceleration as a risk indicator in its Responsible Scaling Policy: if the pace of AI progress doubles because of automated AI research, that should trigger a warning level.

That's what METR set out to verify. The nonprofit studies, among other things, whether AI developers could lose control of their systems. From August 12 to October 29, 2026, a team of 2.5 full-time staffers received what they describe as unprecedented access to benchmark results from unreleased models, training metrics, employee surveys, and internal analyses, along with roughly 15 interviews with Anthropic researchers. METR took no money from Anthropic but worked on Anthropic devices for security reasons.

Anthropic CEO Dario Amodei had recently committed to bringing external auditors into the company. In October, Anthropic then announced a collaboration with METR involving "employee-like access". The current audit is a separate project with a narrower scope, according to METR.

Two years of progress packed into one

Since October 2025, Anthropic's model test scores have been climbing about 1.9 times faster than they did between March 2024 and October 2025, according to METR. The measurement is based on the AECI, a composite metric that bundles many benchmarks together. Anthropic achieved in one year what previously took two, roughly comparable to the jump from Claude 2.1 to Sonnet 4.5. The analysis also included three unreleased models that finished training between July and October 2026. Before October 2025, Anthropic's models gained about 15 AECI points per year; since then, the rate has hit 29.

METR is confident the acceleration isn't driven by faster growth in training compute. Instead, the auditors estimate that about 80 percent of the current speedup comes from AI being used in Anthropic's own research. One employee said in the interviews: "Now only bottlenecked on ideas & compute, no longer bottlenecked on code."

Others reported that experiments that used to take one to two days to set up are now "~free." In informal Slack polls from October 2026, employees said on average that the latest internal model made them roughly twelve times more productive. METR treats these self-assessments as "very loose evidence," calling them unrepresentative and likely to overstate the actual gains.

Self-reported productivity gains rose from 1.2x with Claude 3.7 Sonnet to 12x with internal Model 4. | Image: METR[ AI isn't doing fully autonomous research yet. The acceleration comes mainly from AI acting as an assistant, but there are "signs of life" that independent contributions could play a bigger role soon. METR found no bottlenecks that would slow the pace over the next twelve months, which means there's a "significant risk" that the acceleration will keep growing over the coming year.

Big uncertainty and dissenting voices

METR also stresses how imprecise the results are. The true acceleration factor could be as low as 1.3 or as high as 3. When models from OpenAI, Google, and xAI are factored in, the industry-wide number drops to just 1.4, possibly because Anthropic lagged behind competitors in 2025 and a catch-up effect is skewing the figures.

Views inside Anthropic aren't uniform either. One researcher pegs AI's share of the productivity gains at just 15 to 20 percent and distrusts colleagues' self-reports. Another considers new hires the single most important factor of recent months.

Redacted evidence weakens the audit

Anthropic had unilateral redaction rights over all non-public information and used them heavily. According to METR, the majority of the evidence the auditors rated as most informative was stripped out, including forecasts for future models, breakdowns by training steps, and the bottleneck analysis. Anthropic describes the redactions as "aimed at removing confidential information that could accelerate other AI developers."

The result is that the public has to rely on METR's judgment "much more than would be ideal," the organization writes, and any talk of transparency around the evidence is off the table. One graph in METR's report accordingly features the words "FAKE DATA" in large letters.

Anthropic pulled the plug on a compute allocation audit

In a companion review, METR also examines Anthropic's risk report from August 2026. In that report, the company publicly stated the acceleration was "less than a factor of 2." The internal, unredacted version implies a value of 1.97, though, and Anthropic itself calls it "plausible" that the threshold has been crossed. METR considers it possible that the warning level was already reached on July 15 and accuses the public report of understating both the acceleration and the uncertainty around it.

METR currently shares Anthropic's assessment that the acute catastrophic risk is "low," but the organization can't rule out another sharp acceleration over the next six months, which could quickly push the risk rating higher.

METR was originally supposed to also verify how Anthropic splits its compute between capability research and safety research. Anthropic canceled that part of the project midway through, saying it wanted to focus on risk assessment first and revisit the compute audit later.

Daniel Kokotajlo, the researcher behind the "AI 2027" scenario, responded sharply on X, arguing that auditing the compute allocation is the foundation for making commitments to slow down verifiable in the first place.

AI News Without the Hype – Curated by Humans

Read on for the full picture. Subscribe for hype-free coverage.

Models & LabsUnited States

Partnering with Hacktron to find vulnerabilities in OpenAI’s systems

Credited to OpenAI · openai.com21h ago

Useful

Today we’re announcing a partnership with Hacktron to find vulnerabilities in OpenAI’s systems. Our engineers and Hacktron’s security researchers will choose high-priority areas to test together, with a focus on how our systems interact with third-party services. As part of the partnership, we’re expanding Hacktron’s access to our frontier models through Daybreak Red.

As we work to accelerate cyber defense with frontier AI, we’re applying the same urgency to finding and fixing vulnerabilities in our own systems. Earlier this week, we announced that Cathedral’s researchers will work alongside our teams in our next security sprint. We also raised our bug bounty rewards to give independent researchers more reason to pursue vulnerabilities our own testing may have missed.

As AI agents connect services in new ways, that combination of skills becomes even more valuable. Working alongside Hacktron will help us test those connections more deeply and use what we learn to strengthen our defenses.

“ The future of cybersecurity won’t be defined by who has the largest security team, but by who can best combine human creativity with AI-driven capabilities. OpenAI is setting a strong example by putting frontier cyber models in the hands of security researchers, enabling them to uncover complex vulnerabilities at the boundaries between interconnected systems. We’re excited to deepen our partnership with OpenAI, combining our offensive security expertise with their frontier models to help secure OpenAI and the broader software ecosystem. ”

Keep reading

Original · OpenAI

Front

Models & Labs · United States

AI’s black box may not be so black after all

Credited to TechCentral · techcentral.co.za · 20h ago

A claim that AI models are far easier to understand than researchers assumed has drawn wide attention in the AI safety field, though South African academics caution that the findings are unverified and not yet peer reviewed.

Nick Cammarata, a researcher at AI start-up Goodfire who previously worked at OpenAI, makes the case in Neural networks are already clean, billed as “an informal research essay and working hypothesis”.

“The canonical evidence of polysemanticity and superposition that convinced much of the field, including me, that networks pack concepts arbitrarily was misread,” he writes. Those terms describe the idea that individual neurons juggle several unrelated concepts at once.

He told MIT Technology Review: “We were looking at it wrong.”

Steven Boykey Sidley, visiting professor at Rhodes University, said AI’s black box nature, which even its builders cannot untangle, underpins much of the anxiety about it.

The Cammarata paper suggests that it is not that complicated inside the AI neural network – we have just been looking at it through the wrong lens. Steven Boykey Sidley

“This has been unnerving, because if you can’t understand how AI thinks, then how can you prevent it from doing bad stuff, like urging someone to commit suicide? That is at the heart of what is called AI alignment,” he told TechCentral.

“The Cammarata paper suggests that it is not that complicated inside the AI neural network – we have just been looking at it through the wrong lens. His partial and (as yet) unverified results are tantalising, because they hold out the promise that we can make AI our subservient tool, rather than the uncontrollable and unruly child that it now is.”

Cats, cars and a V

The essay revisits a famous case in interpretability, the field that tries to work out what happens inside AI models. Neuron 4e:55 – a unit that activates when it detects a particular feature – in Inception v1, a 2014 Google image-recognition model, responds to both cats and cars. In Zoom in, a 2020 paper Cammarata co-wrote in work led by Chris Olah, now head of interpretability research at Anthropic, which he co-founded, it was held up as a textbook case of a neuron mixing unrelated concepts.

Cammarata now argues that what links them is a shared V shape: the pointed face of a cat, the front of a car seen at an angle. The neuron also fires for a pool table’s corner and a bullet train’s nose, while round shapes push it the other way. A 30 October revision to the essay makes clear that the neuron still detects cats and cars, with the V shape as the common feature.

“It probably took me five seconds,” he told MIT Technology Review, having studied the model “for probably thousands of hours”.

If most neurons have one main job, he argues, researchers can inspect models without first training a second model to untangle them.

Preliminary audits of Alibaba’s Qwen3-32B model suggest that roughly 90% of “neuron poles” – the positive and negative ends of a neuron’s activity – have one main job, about 7% have two related jobs and 2-4% are not yet understood, he says, though the figures depend on thresholds. A project he says he started at OpenAI in 2023 found only about 1.7% of neurons interpretable “by our most generous measure”.

Not yet peer reviewed

Prof Benjamin Rosman, founding director of the Wits Mind Institute, named among Time’s 100 most influential people in AI in 2025, called it “an interesting and potentially very encouraging development for AI safety”.

“However, the research has not yet been peer reviewed, and it is far too early to draw strong conclusions about what this means for AI safety,” he said.

Others are more sceptical. “This is interesting work, but I don’t think it’s a significant update to the big picture of interpretability,” Google DeepMind’s Neel Nanda wrote on X. Jacob Hilton of the Alignment Research Center added: “For a full understanding, we need to go beyond neurons, and explain how they connect together to form circuits.”