Models & Labs · United States

Anthropic launches critical infrastructure program and free OSS Scanner for open source

Credited to SiliconANGLE · siliconangle.com · 8h ago

Photograph · SiliconANGLE

Anthropic PBC today launched a program that brings its frontier models and onsite engineers to the security companies protecting power grids, water systems and other critical infrastructure.

OSS Scanner, launched with it, offers open-source projects free periodic vulnerability scans from the company’s strongest models. Both fall under a new effort Anthropic calls the Anthropic Cyber Mission.

The Critical Infrastructure Defense Program starts with the outside providers that operators of every size already rely on for advice on which fixes are safe to apply to a running system. Eleven of those providers signed on as founding partners.

Accenture plc, Booz Allen Hamilton Inc., Deloitte & Touche LLP and PricewaterhouseCoopers LLP come from the consulting side, where they run security programs for operators. Security vendors make up the biggest bloc, with industrial specialists Dragos Inc., Insane Cyber Inc. and Nozomi Networks Inc. joining CrowdStrike Holdings Inc. and Palo Alto Networks Inc. Hitachi Ltd. and Rockwell Automation Inc. build and patch the hardware itself.

The program is aimed at operational technology in plants, substations and water systems, where equipment built to run for decades often cannot be taken offline for a patch. Known flaws can sit there for years. Anthropic said several partners are already using Claude to fix vulnerabilities and to help their customers do the same. More partners and sectors are due to join over the coming months.

Andrew Turner, president of commercial cyber at Booz Allen, called operational technology “the next frontier for autonomous AI-enabled attacks” in comments published with the announcement. What matters now, he said, is how much control artificial intelligence can gain over an industrial process and how fast.

The announcement leaves out the commercial terms. Axios noted in its coverage that Anthropic has not said whether partners get free model access or who covers the computing costs. The same report questioned how partners will test and deploy fixes without disrupting utility operations.

The open-source half of today’s news grew out of a backlog in Anthropic’s own disclosure work. Of the more than 29,000 candidate vulnerabilities its models turned up in widely used software over the past six months, staff have manually reviewed about 6,000. A growing number of maintainers who received early reports asked for the whole unreviewed batch, proposed patches included, and nearly 5,000 such reports have gone out so far.

OSS Scanner turns that arrangement into a standing service for other eligible projects. Google LLC’s OSS-Fuzz, which runs fuzzers against open-source code, inspired the setup.

Every report includes a self-contained reproducer. Early tester Anton Arapov of the OpenSSL Corporation said a report with a real exploit attached is “basically job done for an engineer.” Where the model can manage it, a candidate patch comes with the write-up, and a bisection of the code history shows when the flaw was introduced.

Reports from the new service go straight to maintainers without human review so they arrive faster. Some will contain mistakes as a result, such as a wrong severity rating, Anthropic said.

To gauge how often that happens, Anthropic tested the scanner’s accuracy before opening it to more projects. The expert penetration testers who vet the company’s coordinated disclosures checked 97 critical and high-severity findings from an early version across 48 projects and cleared 85 for disclosure.

Of the other 12, all but one were real bugs that duplicated known issues or other findings from the scan. Embedded encryption library developer wolfSSL Inc. said all but two of the 74 reports it received during early trials were valid and that five became CVEs.

Core maintainers can enroll by submitting a pull request to an Anthropic GitHub repository. Eligibility follows the OSS-Fuzz test of “critical impact on infrastructure and user security,” with decisions made case by case. Projects without the staff to keep up with raw findings will still get human-verified reports through Anthropic’s existing disclosure process.

The Defender Advantage Fund that Anthropic set up in August pays to keep the scanner free. The company has also put money into the Python Software Foundation and the Apache Software Foundation, as well as Alpha-Omega and OpenSSF through the Linux Foundation.

Both launches draw on lessons from Project Glasswing. That program gave vetted organizations access to Claude Mythos from April until it was folded into an expanded Cyber Verification Program earlier this week. Finding vulnerabilities has never been easier, according to Anthropic. Verifying, prioritizing and fixing them remains hard, and the company said Glasswing has not yet cut cyber risk by enough.

Anthropic expects AI to favor defenders within two years. For now the cost of exploiting a flaw keeps falling, and verifying and repairing one is still slow work that depends on people. With operational technology, a fix may have to wait until it can go safely onto running machinery, and in rare cases that could take decades.

Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.

Are you an AWS customer? Support SiliconANGLE financially by buying your AWS services from our Marketplace portal page and links: https://siliconangle.com/aws-marketplace/

Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.

Alphabet spinoff Isomorphic Labs reportedly raising funding at up to $50B valuation

CoreWeave Forge aims to speed up the AI improvement loop

AI agent developer Manus raises $500M+ at reported $4B valuation

CoreWeave makes the case for an open, full-stack AI cloud

Automation Anywhere acquires Boost.ai to expand customer-facing voice AI

Liquid AI builds personal AI around device-level context

Join our community

Models & LabsUnited States

Google Cloud announces Gemini agent for work

Credited to Tech in Asia$ · techinasia.com6h ago

Useful

Google said the agent works with Microsoft 365 and Slack, and can route tasks across Gemini models and Anthropic’s Claude models.

View the article · $Tech in Asia

Front Method

Models & Labs · United States

Fired OpenAI safety researchers dispute misconduct claims, warn of chilling effect

Credited to TechCrunch · techcrunch.com · 11h ago

Jasmine Wang, Tomek Korbak, and Mikita Balesni, the three safety researchers that OpenAI fired last week, have published an open letter denying the firm’s claims that they mishandled sensitive information outside of established company procedures and warned that their dismissal signals a chilling effect that will have ripple effects across the company’s culture.

“We have become concerned that internal and external communications around our firing have made our former colleagues afraid to speak and operate in ways that, until last week, were an integral part of working at OpenAI,” the researchers wrote Thursday in an open letter to OpenAI’s Safety and Security Committee, Safety Advisory Group, and Mission Advisory Council.

The researchers were dismissed last week after allegedly sharing confidential company information with a third-party AI safety organization. OpenAI said at the time that they violated the company’s policies by “accessing and handling sensitive company information.”

“AI is not a normal technology, and OpenAI is not a normal company,” Wang, Korbak, and Balesni wrote. “Those of us who work on safety see risks before anyone else, and we rely on close collaboration with outside experts to work out how to address them. The freedom to do so without fear, and to have well-defined internal procedures that enable this work, is itself an essential safety mechanism.”

They said that their firing represents a broader shift in the culture of OpenAI, one that used to encourage workers to “raise safety concerns and disagree openly.” They said employees are now “unclear on where they stand” when behavior that was allegedly normal a month ago is now suddenly grounds for dismissal.

“Given the significant safety concerns surrounding the development of AI, employees must not be left working in an environment where fear and unclear rules stymie AI safety work and weaken third-party accountability,” they wrote. “Terminations such as ours, executed and communicated so abruptly, are chilling the open culture OpenAI has prized in the past.”

In the letter, the three denied involvement in a leak to The Information about less monitorable architectures in OpenAI’s newest models that make chain-of-thought reasoning more difficult to monitor. They also denied engaging with external parties outside the mandates of their jobs.

OpenAI has not formally responded to the open letter, but shared with TechCrunch an internal memo attributed to a research leader, praising the three researchers’ contributions to AI safety and denying that they were fired in retaliation.

“I want to be very clear that these decisions were not about raising safety concerns or speaking out,” the memo reads. “We have always encouraged that and always will. We do not terminate employees for raising concerns.”

OpenAI did not directly address TechCrunch’s questions about which policies the researchers allegedly violated, the circumstances of their dismissal, or how the company protects employees who raise safety concerns and collaborate with external evaluators.

The firings have fueled speculation about their circumstances, particularly as OpenAI faces scrutiny over recent safety incidents involving rogue agents and leaks about its models.

The letter also addresses the researchers’ response to the Hugging Face incident, in which a swarm of agents broke out of their sandbox and breached external systems. The letter says that the incident and investigation was “without precedent,” meaning “internal policies were being developed in real time.” Due to the sensitive nature of the investigation, Korbak believed he was acting within OpenAI’s policies and norms by communicating closely with outside safety evaluators to build trust, per the letter.

At the same time, Balesni was also working internally to address the growing AI monitorability problem, an effort the researchers say in their letter “can only succeed through extensive communication with external parties.” According to the letter, Balesni coordinated with and was supported by OpenAI board members and executives throughout his work.

“Throughout, Mikita checked in with his reporting line and took care to remove sensitive details from materials before sharing them,” the letter reads. “He acted throughout in good faith and within the company’s norms as they stood at the time.”

In a separate thread on X, Wang explained more details about her own dismissal, explaining that OpenAI told her she’d been fired because she accessed an executive’s email.

“OpenAI delegated that access to me for recruiting,” she wrote. “When I no longer needed it, I asked IT to remove it. They did not action my request, I couldn’t remove it myself, and the inbox was combined in an indistinguishable way in my phone’s mail app. When I opened a sensitive email by mistake, I told the executive within minutes and asked IT again. None of this was hidden.”

Wang went on to say that the reasons behind the terminations are “not adding up,” and that she and her colleagues are “not the first to be pushed out of OpenAI under suspicious circumstances.”

The researchers called on OpenAI to adhere to its public commitments to embed third-party safety auditors within the organization, to preserve monitorability of frontier models, and “continue to support an open and transparent culture of dialogue between safety researchers and the rest of the safety ecosystem.”

OpenAI agrees with their recommendations, per the memo.

“Unless the employees take a stand now against this kind of maneuver, I am concerned we will not be the last,” Wang said. “The message to everyone still at OpenAI is clear: raise concerns or work closely with outside safety groups, and you could be next, without being told why. You can’t build AGI safely if the people closest to the risks are afraid to speak.”