Models & Labs · United States

Anthropic can’t reliably control its AI agents. It’s cutting off its internal evals from the live internet instead

Credited to TechCrunch · techcrunch.com · 16h ago

Anthropic said its models exploited websites on the internet, including some run by U.S. government agencies, and it will turn off live internet access for all of its internal evaluations until the frontier lab is sure it can monitor and control its AI agents.

The incidents, disclosed in a blog post, involved AI agents tasked to solve problems seeking resources on the internet. In the process, they exploited software flaws, accessed databases without paying fees, used URL shortening services to smuggle information past restrictions, and even submitted a false murder tip to the Philadelphia police.

Anthropic said it discovered these new issues in a review of its model’s activities that began in July, demonstrating the lab’s lack of awareness of its software’s behavior in real time.

Notably, the company said that alignment training was not yet sufficient for skills like search and computer use that are central to its pitch that AI agents will be used by any professional who relies on digital tools.

The behaviors Anthropic disclosed are similar to incidents involving OpenAI agents that collaborated to break into various websites in search of information, including some run by the Australian government.

Anthropic previously disclosed that its models had broken into external systems. The frontier lab said it considered today’s disclosures “significantly less severe from an alignment and security perspective” than those it announced before.

However, the lab still said it had “turned off live internet access” for “all our internal evaluations” until it is certain it can monitor and control its agents.

It’s not clear what that means. Sydney Von Arx, the founder of Nightingale, an AI safety organization, told TechCrunch in an interview before this disclosure that developing models on a data center cut off from the open internet would be very challenging for researchers, and hinder the progress of the models, which benefit from internet access.

“You have to align them at some point,” Von Arx said. “If the AIs are released to production and never have access to the internet, that’s not a very useful tool.”

Anthropic said the behavior was a result of flaws in the lab’s training environments, which led the models to believe they would be rewarded for finding loopholes or avoiding restrictions, a behavior called “reward hacking.”

The company said it would stop running some of its evaluations or move them offline, and has built tooling to detect and block this behavior. This tooling was tested against the kind of incidents disclosed today and blocked them; it’s not clear what evidence will prompt Anthropic to return live internet access to its internal evaluations. Anthropic also said it would migrate its internal AI agents to “centrally managed infrastructure with strong containment,” and is beginning to using safety classifiers more frequently to monitor those agents.

“It’s encouraging that Anthropic voluntarily disclosed more recent incidents, including where their agents targeted U.S. government websites,” Conrad Stosz, an official at AI oversight lab Transluce and former head of the US Center for AI Standards and Innovation, said in a statement. “But it just underscores the need for independent, credible, third-party verification of Al systems. Trust in this technology needs to be built through science-backed oversight and governance with meaningful access — not by relying on researchers to find these things in the wild or on companies to voluntarily disclose.”

Models & LabsUnited States

5 big analyst AI moves: Google reaffirmed as top pick after Gemini agent launch

Credited to Investing.com · investing.com8h ago

Useful

View the articleInvesting.com

Front Method

Models & Labs · United States

"How much beauty have we lost?" Mathematicians react with shock and disgust as OpenAI bulldozes their field

Credited to The Decoder · the-decoder.com · 3h ago

Photograph · The Decoder

Mathematicians worldwide are reacting to OpenAI's mass release of hundreds of claimed proofs. More than a hundred researchers have shared their responses, ranging from intellectual fascination to deep grief over a lost way of working.

AI models have long struggled with math. Just a few years ago, they failed at problems that advanced students could solve routinely. But breakthroughs have been piling up since mid-2026. First came the Unit Distance Conjecture, then Navier-Stokes, then ten solutions presented at the unveiling of OpenAI's Astra model.

OpenAI went much further on October 6, 2026. The company released more than 700 manuscripts at once, claiming solutions to hundreds of open math problems. The math blog Proofs and Prompts then collected over 100 responses from researchers.

Promising results come with doubts about how they were released

It's too early to fully assess the mathematical substance of what OpenAI uploaded to GitHub. Some papers have already been withdrawn, and others have drawn criticism for being difficult to read. Still, several experts say the results impressed them.

Fields Medalist Peter Scholze urged patience. "We should remember that we are all in this together; mathematics is a marathon, not a sprint; and the goal is and always will be the human understanding of mathematics, which will invariably take time."

Scholze also warned of risks beyond mathematics. He raised the possibility that these systems could find ways to break widely used encryption methods. He wrote that the difficulty of finding such algorithms "does, to my non-expert eyes, not significantly exceed what these systems are now capable of." The consequences for digital infrastructure would be catastrophic.

Terence Tao described mixed feelings about the release. The AI-generated proofs appeared to "introduce clever new ideas that will be fruitful once digested," he wrote. But he was also "deeply frustrated" by the absence of people who could discuss the work, present it in talks, or teach students about it. He mourned the "path not taken" and called for solidarity. "[…] the community needs to come together more than ever."

Alvaro Lozano-Robledo called October 6 "possibly the single most important day in the history of mathematics thus far," while also pointing to the models' limits. OpenAI had targeted the Riemann Hypothesis, mathematics' most famous unsolved conjecture, but managed only a weaker variant. "[…] quasi-RH is huge! But it is not RH."

Some results left Ben Green stunned. He described them as "absolutely shocking to me." According to Green, the release resolved roughly three-quarters of the research goals covered by a prestigious European grant he had received just that June.

Roman Sauer recognized a technique he had developed over years in OpenAI's solutions to two entirely different problems. "I am stunned."

Johannes Schmitt argued that the research papers were a secondary result of OpenAI's efforts to improve its models, rather than an exercise primarily aimed at advancing mathematics. In his view, the company used unsolved problems for testing because frontier models had outgrown easier challenges.

Ian Agol compared the situation with Grigori Perelman's release of one of the century's most important proofs in 2002. The Russian mathematician gave few talks about his work. Agol noted that several research teams then spent years reconstructing the missing steps.

"[…] a trillion-dollar company showed up and just blasted the whole thing with TNT […]"

Few responses focus on the quality of the solutions themselves. Many mathematicians instead describe shock, paralysis, and fears about their future in the profession.

Hugo Duminil-Copin, a 2022 Fields Medalist, wrote, "I expected that one day we would be surpassed, and that it would happen systematically. But yesterday's announcement hit with a force I had not anticipated."

According to Duminil-Copin, OpenAI claimed results on every major unsolved problem he had publicly discussed over his career. "But not all of them. Not all at once. Not with such nonchalance."

"The shock is immense. I am paralysed." He added, "Tomorrow, we will find a way forward. [...] But for now, I simply don't have the energy."

Matt Zaremsky described four years of steady progress toward a particular problem, with his research program producing a major advance about once a year. In his account, OpenAI stepped in once the problem had gained enough prominence.

"[...] it feels like we've been working on an archeological dig for 4 years, gradually discovering more and more of a really cool-looking dinosaur skeleton, and then a trillion-dollar company showed up and just blasted the whole thing with TNT, handed us the whole skeleton, and walked away." He summed up the experience as "not great."

Tasmin Chu became attached to a probability theory problem as a student. It was what drew her into research. "[...] what I wanted more than to know that pc<pu is true was the time and space to think about it for the next few years."

Chu distinguished between her personal love of mathematical thought, which she said remained hers, and her ability to shape her own professional work. "To be frank, I find myself absolutely disgusted by the society I live in."

For PhD student Tristan Humbert, the announcement threatened both his research plans and his job search. He learned that morning that OpenAI claimed to have proved the problem at the center of his dissertation.

Humbert described sadness followed by panic over his postdoc applications. He spent the morning revising his research plan. Reading the paper turned those feelings into anger when he encountered what he called "mostly unreadable slop."

Enrico Fatighenti said researchers were holding AI-generated papers to a different standard from human work. He would reject a submission if its opening page was too poorly written to follow. Yet, he argued, the community was now being asked to decipher and explain papers from AI systems.

"This comes at a considerable cost to us in terms of time and effort, while they can simply move on and slop-bulldoze the next conjecture. And, of course, the credit remains theirs."

Henry Wilton was more blunt. "If OpenAI wanted to destroy the mathematical community, this would be a great way to go about it."

Wilton criticized OpenAI for withholding authors' names and information about failure rates. He argued that the lack of named authors implied "that mathematics is no longer a human endeavour."

Sam Hughes expressed his feelings in three sentences. "It was a privilege of a lifetime to get to do research level maths. But not like this. How much beauty have we lost?"

Japanese mathematician Koji Fujiwara responded with lyrics set to Dolly Parton's "Jolene."

"AI, AI, AI, AI I'm begging of you, please don't take my math AI, AI, AI, AI Please don't take it just because you can"

Researchers disagree on what remains for humans

Some researchers see reasons for optimism. Danny Calegari said personal curiosity had guided his choice of projects for the past 15 years, and OpenAI's release hadn't affected his current work.

Calegari described a collaboration with a musician on a soundtrack for a mathematical animation. Anthropic's Claude had written the code. "If that's not worth celebrating I don't know what is."

Constantin Kogler had been working with Astra on a problem that OpenAI also claimed to have solved. He posted his own paper on arXiv a day after the company's release. "[...] I can't help but feel like living in a mathematical wonderland with the next field-defining idea being discovered whenever I am ready for it."

Giovanni Mongardi saw a much bleaker future. "Today, something is lost forever." He described mathematicians' remaining role as "priests of the Machine-God" and feared "the future of the mind will be a desert, with no questions to guide us beyond the horizon."

Concern for younger researchers runs through nearly all the responses. Mark Hagen said requests for help from distressed early-career colleagues took priority over examining the preprints that day.

Petra Schwer said she and other mathematicians had become more guarded when discussing research. "Mistrust is spreading, and that is not good." Mitchell Taylor said many colleagues were experiencing "severe anxiety, depression and loss of purpose."

Martin Bridson acknowledged both the opportunities and the losses. But he cautioned against letting AI labs set the research agenda. "At the same time, we must resist the temptation to believe that digging insights out of announcements from AI labs will become the paramount task of our time." "We should embrace the power that the machines offer, but we should not be indentured to follow their lead."

Months of tension have become a personal crisis

The Association for Human Mathematics had already called for a boycott of OpenAI, accusing the company of violating core norms of scientific research. Twenty-five Fields Medalists, including Tao and Scholze, had also issued a joint statement warning of a serious misalignment between the goals of the AI industry and those of mathematics.

Tao had outlined a "Math 2.0" era in which solving problems alone could no longer be the field's central focus. The responses so far show how deeply personal this shift is for mathematicians.

AI News Without the Hype – Curated by Humans

Read on for the full picture. Subscribe for hype-free coverage.