Anthropic gives six METR evaluators access to internal systems, with publication rights
Anthropic has embedded six METR evaluators in its offices, giving them company hardware and access to internal systems comparable to its own risk teams. METR can publish its findings without Anthropic’s approval, although the company can require redactions for customer data, privileged information and model architecture details, according to Ars Technica and WebProNews.
Bottom line — METR can publish its assessment without Anthropic’s veto, but the company retains limited redaction rights.
Go deeper 6
-
Anthropic said it is not paying METR; the nonprofit is funded by individual and philanthropic donations and government bodies, Ars Technica reported.
-
METR can publicly describe redactions and challenge whether they were justified, WebProNews reported, provided it does not disclose the withheld information.
-
WebProNews says this is the highest of three tiers in Anthropic’s evaluator system; lower tiers have narrower access or require approval to publish.
-
Anthropic compared embedded evaluators to resident bank examiners, while acknowledging the analogy is not exact, WebProNews reported.
-
Anthropic said employees are encouraged to share information with the METR team as they would with colleagues working on risk, Ars Technica reported.
-
Anthropic is also working with Accenture and SecureBio on safety testing, Ars Technica reported.