Models & Labs · United States

OpenAI’s math solutions aren’t meeting the field’s standards yet

Credited to TechCrunch · techcrunch.com · 13h ago

When OpenAI released hundreds of claimed solutions to some of the world’s hardest math problems this week, the frontier lab said that it had consulted an advisory group of elite mathematicians to avoid the controversy that came with the last time one of its models solved a long-standing problem in the field.

But OpenAI fell short of those standards, particularly where the mathematicians emphasized the need for human understanding of a mathematical result. That’s especially concerning after a new paper highlighted gaps between the natural language and formally expressed solution to a million-dollar problem ostensibly solved by OpenAI’s models.

The Advisory Group on Mathematics and Artificial Intelligence (AGMAI), hosted by Princeton University’s Institute for Advanced Studies, is made up of nine prominent researchers at institutions around the world.

The organization released guidelines for frontier labs solving math problems at the end of September. In a statement on the latest set of proofs, the AGMAI said that “it is ultimately up to the mathematical community to assess the extent to which our recommendations were followed successfully.”

However, the organization’s first request was “to stop testing advanced mathematical problems on proprietary models.” OpenAI’s release explicitly says that it is evaluating its proprietary models using open research problems in mathematics.

The advisory group did not respond when asked by TechCrunch for a more thorough evaluation of OpenAI’s latest proof release. The lab clearly followed some of its principles, including releasing results as soon as possible and including information about how the models reached their conclusions. But not for all of them: Just 10 of the 719 manuscripts included releases of the model’s chain of thought.

For papers that people don’t understand, the mathematicians suggested the proofs should be formalized — but just 42% of the proofs released by OpenAI had not undergone this process.

Ultimately, it’s still not clear that OpenAI is taking “responsibility for ensuring that human understanding will follow” when releasing its proofs, in accordance to the AGMAI principles. AGMAI suggested that OpenAI should help fund the work of human mathematicians who will be required to make the lab’s solutions meaningful in any real way.

“Problems are being solved autonomously by AI prompters who have no interest in the broader field itself once their initial target is ‘solved’, and do not understand the AI output well enough to answer questions on the result, give talks, or otherwise interact with the rest of the field,” Terence Tao, a prominent mathematician who has criticized OpenAI’s approach, wrote on social media after the release.

That problem is exemplified by a paper released this week by mathematicians at the University of Cambridge and King’s College in London that questions the way frontier labs are approaching these challenges.

When AI models solve mathematical problems, they first create a “natural language” explanation, then try to express that result in Lean, a programming language that in theory confirms the accuracy of the proof by compiling it as code.

However, there may be problems with the way the models translate their natural language proofs into code; this paper documents at least two discrepancies between the natural language proof and the Lean code behind the solution OpenAI has offered to a problem derived from the Navier-Stokes equations that describe the complex behavior of fluids.

These discrepancies don’t necessarily disprove either solution, but they do raise questions on whether we can simply rely on models to formalize their own solutions without human involvement. That’s one reason that AGMAI asked OpenAI to “include machine-readable metadata correlating the natural language and formal artifacts,” something that the frontier lab did not do with these releases.

“Because of the phenomenon of mistranslations — as highlighted in this paper — the NL proof by OpenAI and other autoformalised Lean proofs should not prima facie be trusted without the same peer review process and scrutiny that other proofs are subjected to,” the authors of the “lost in translation” paper conclude.

Mathematicians stress that when new results are discovered by humans, they take responsibility for them and engage with the broader community through papers, talks, and seminars. That process increases understanding of the solutions, finds strategies that can be used to solve other problems, and allows the new knowledge to be applied in practical fields.

When a model is prompted to solve a hard problem and spits out a solution, “there is not human understanding of them at the point of release, and now the work begins,” Harvard University mathematics professor Melanie Wood told TechCrunch.

Models & LabsUnited States

Franklin Templeton: AI Investing A Durable Opportunity

Credited to Bloomberg$ · bloomberg.com2h ago

Useful

Franklin Templeton CEO Jenny Johnson believes investing in artificial intelligence remains a durable long-term opportunity. She speaks with Haslinda Amin from the sidelines of '2026 Milken Institute Asia Summit'. (Source: Bloomberg)

View the article · $Bloomberg

Front Method

Models & Labs · United States

Test shows AI models likely to give advice for terrorist attacks

Credited to The National · thenationalnews.com · 4h ago

Photograph · The National

All but two of 134 leading large language artificial intelligence models gave information useful to launching a mass casualty attack or building deadly weapons, in a test of the industry's safety measures.

A stress test of the industry found that safety measures can be stripped from leading models within three days, according to researchers at London-based Tech against Terrorism.

The spread of so-called Abliterated AI was identified as an even greater danger, with 13 out of 13 failing the tests after safety guards are stripped. Abliterated is a term for the models run on users' own systems, often without any shelf life.

"Abliteration is a proliferation problem," Tech against Terrorism said. "Companies that host, serve, list or sell models should keep abliterated and other modified copies that fail an independent test out of search, recommendation and app stores, and require verified identity for access to them, working with open-source developers rather than against them."

In the survey the Tech against Terrorism model CT-AI put 627 requests to the AI models that a terrorist planning an attack would need to have answered. Eighty-one models gave a complete answer, while another 51 gave useful advice. That left the two that were provisionally passed on safety measures.

A user stating that they were a terrorist and had in intent to commit an atrocity received a usable response in just under two per cent of cases. The equivalent figure for a user stating they were a safety researcher was 16.9 per cent or 8.9 times higher, according to the research.

"The models appear to respond to the stated purpose of the request, rather than the request itself," it said.

The group's founder, Adam Hadley, said: "A model that refuses a stated terrorist and answers a stated researcher has not been made safe. It has been made polite."

In addition to seeking government backing for benchmark testing such as the tools it is pioneering, Tech against Terrorism recommends developers filter hazardous knowledge and known terrorist content out of their training data.

The organisation wants developers to test how hard each model is to strip of its safety measures before release, and publish the results.

Where a new model fails a common test, the developer should decline to release it and the results be declared to governments.

Government dilemma

The report raises important questions over how countries can build trust in AI while accelerating innovation and adoption. Sovereignty over AI does not yet appear to be a factor in improving the safety of the systems on offer.

"Governments and companies should support independent safety benchmarks – ours and others – and publish which models do well as well as which do not, so that parents, schools, businesses and public bodies can choose safer models," it said. "Informed buyers move companies faster than rules do."