Analysis
Anthropic began giving six evaluators from METR, the independent AI-safety nonprofit, what it calls "employee-like access" to its internal systems on Oct. 15, according to Ars Technica. The arrangement hands METR's team company hardware, office space and access to internal tools to vet Anthropic's unreleased models and agents before they ship -- a level of outside scrutiny no other frontier AI lab has granted.
The move traces to a September essay from CEO Dario Amodei, who proposed embedding outside evaluators inside AI labs as a way to close the gap between what regulators and watchdogs understand about frontier systems and what the labs themselves know. ProPakistani reported that METR's access specifically excludes customer data, attorney-client-privileged material and the deepest model-architecture details, while otherwise approximating what Anthropic's own internal risk-assessment staff can see.
Anthropic's arrangement sets up a three-tier system for outside evaluators. METR currently sits alone at the top tier, with full access and the ability to publish without Anthropic's prior approval -- though Anthropic can still require redactions in predefined categories, and METR can publicly dispute a redaction it considers unjustified. Lower tiers, including biosecurity-focused evaluator SecureBio, get narrower access and must clear publication with Anthropic first. OpenAI CEO Sam Altman said separately that OpenAI will offer independent evaluators similar access, though without Anthropic's specifics on tiers or redaction rights.
“Anthropic's arrangement sets up a three-tier system for outside evaluators.”
METR already runs third-party capability evaluations for both Anthropic and OpenAI models ahead of major releases, but those engagements have historically been time-boxed testing windows, not standing internal access. Embedding six full-time evaluators inside the company is a structurally different commitment -- closer to an internal audit function than an external one -- and it comes as Anthropic heads toward an IPO that bankers have discussed at valuations as high as $2 trillion, where governance and safety claims face far more investor and regulatory scrutiny than they did as a private company.
## What the headline misses The access Anthropic granted does not make METR independent of Anthropic's control: Anthropic still decides what counts as a redactable category, still owns the hardware METR works on, and still picks which evaluators reach the top tier in the first place. Some commentators have also noted METR's research ties to the same intellectual circles that shaped Anthropic's founding, raising the question of whether this is independent oversight or a more sophisticated version of marking your own homework.
Anthropic has not disclosed what happens if METR's findings and Anthropic's own internal risk assessments materially disagree -- the clause that will decide whether this arrangement is real oversight or a well-staffed public-relations exercise. Pulse has tracked Anthropic's parallel safety posture in its standoff with OpenAI over AI-safety standards this month.
