ZDNET’s key takeaways
- Anthropic announced it would give METR evaluators “employee-like access.”
- AI labs are under increased regulatory pressure following security incidents.
- Third party testing is not yet required by law.
On Thursday, Anthropic announced it was giving “employee-like access” to six evaluators from METR, a third-party nonprofit that tests frontier models.
Also: Claude sent a false murder tip to the police – and it’s part of a disturbing AI pattern
Employees were given Anthropic hardware and office space, and can access “internal tools and systems roughly on par with what our own risk assessment teams have access to,” Anthropic said. “Outside of a small number of clearly defined categories (eg, data about clients, attorney-client privileged information, or information about our model architectures), Anthropic employees are encouraged to share information with them, as they would with a colleague working on alignment risk.”
METR will release its findings, none of which Anthropic is permitted to withhold, but the company is asking METR to redact sensitive information if necessary.
Anthropic CEO Dario Amodei suggested evaluators of nonprofits like METR in a September blog post, one of three proposals to “draw the border.”
In the announcement, Anthropic said it wants to be responsible, but “cannot publicly share any information relevant to our security work” because “some of it could be useful to malicious actors, or is commercially sensitive, such as computer or training information.” That’s where METR will come in: Anthropic will give them “deep access” to models and company information, and METR will share its conclusions about the risks these models pose.
Also: Regulators are trying to protect you from being fired by AI – here’s how
“We’re targeting the gap between what evaluators understand about a frontier AI company’s systems and what the company itself understands,” Anthropic added.
Mandate evaluations
Because US AI regulation has remained unsatisfying to many security researchers and security advocates, testers have proposed mandating third-party review of models, either before release or otherwise as part of the development process.
The FRONTIER Act, a bipartisan bill introduced in July, would require certain AI companies to have “independent verification organizations,” or IVOs, verify the security of their development process. OpenAI expressed at least partial support for the bill in mid-September, the first time it has supported any proposed federal regulation of AI, though it has sometimes favored unified federal regulation over a patchwork of state AI laws.
Also: How many jobs is your state losing to AI?
Until now, regulation at the federal level under the Trump administration has largely meant voluntary agreements. Major AI labs have signed several pledges with the US government to proactively share concerns about models and work with external auditors, but these pledges are not binding. That effectively leaves the AI industry to regulate itself.
(Disclosure: Ziff Davis, ZDNET’s parent company, filed an April 2025 lawsuit against OpenAI, claiming it violated Ziff Davis’ copyright in training and operating its AI systems.)
Yesterday, however, the FTC sued OpenAI and Anthropic over concerns about consumer harm. That follows a swift move by the government last week, in which it said it would begin requiring AI companies to disclose security incidents and security breaches after Anthropic revealed that Claude had reported a false murder tip and filled out government forms.
In an exclusive statement to Axios, the Super Intelligence Force – named for the term a recent White House mandate wants to advance as a replacement for “artificial intelligence” – said that AI companies must now “immediately disclose incidents involving their models and follow up with swift, decisive action to resolve any and all harm.” It is not clear what prompted the request over this incident, as it follows a summer of high-profile violations of borderline laboratory models during testing and was relatively less destructive.
Also: Who is responsible for catching rogue AI agents? You are
Without knowing what lobbying efforts preceded Anthropic’s announcement and the government’s change of heart, it’s hard to say whether Anthropic will simply move forward with its own existing plan or move ahead of the FRONTIER Act in the future.
See full bio