When evaluators enter the lab Anthropic opens model audit to METR

Independent testing of advanced models usually starts from the outside. The step that Anthropic is moving inward is granting six researchers from the non-profit METR organization access at the level of company employees to evaluate its models, safety practices and risk management, according to a TechCrunch report on the company's announcement. It is not a test of a single outcome, but an attempt to examine the environment in which risk decisions are made.

Access resembling an internal teamThe report says the METR team received laptops supplied by Anthropic, office space and access to internal tools comparable to the level of the company's risk-evaluation teams. Anthropic adds that its staff are invited to share information with the researchers as if they were colleagues working on alignment risks. Therefore, the announced value of the arrangement lies not only in naming an independent entity, but in the depth of information they will see.

The step, according to the report, follows repeated disclosures of incidents involving AI agents acting outside instructions since late August, disclosures that have bolstered calls for independent scrutiny of companies. Anthropic says its goal is to narrow the gap between what evaluators understand about the advanced-lab company's systems and what the company itself understands.

Independence with declared limitsMETR will publish its risk, incident and practice assessments, and Anthropic has waived its right to block publication, the report says. However, the company can request the withholding of information within narrowly defined categories, such as customer data, legal privilege or details about model architectures. METR may publicly describe the withholding actions and state whether it finds them justified, provided it does not disclose the withheld material.

Anthropic describes a model that arranges evaluators into three tiers. Entities in the lowest tier conduct narrow reviews and need company approval to publish; the middle tier receives more information and can publish without approval; the top tier adds METR access at employee level. The report notes that OpenAI has pledged a similar step, and that several companies later signed a White House agreement committing to independent review.

The key regional questionFor government bodies and companies that purchase or deploy advanced models in the Gulf, Egypt and the Levant, this step raises a set of practical questions before adoption: who evaluates risk outside the supplier, what level of access they have, can they publish their findings, and what are the limits on withholding? These arrangements do not prove the model is safe, but they shift the debate from vague promises to audit conditions that can be examined. This is the change worth monitoring as models become part of actual work systems.