Anthropic AI speed doubles amid IPO
An independent research group found that the San Francisco firm is developing its models twice as fast as last year, contradicting its CEO's recent calls for caution.
By Dominic Foster ·
Anthropic is developing its artificial intelligence systems around twice as quickly as it was a year ago, according to a new analysis released Thursday by an independent monitoring group.
The finding comes from the Model Evaluation and Threat Research, a nonprofit based in Berkeley, California. The group was invited by Anthropic to review internal data and interview staff as the company prepares for its Wall Street debut. This timing coincides with growing debate over whether AI models can elude human control with catastrophic consequences.
The analysis challenges the public stance of Anthropic CEO Dario Amodei. In a Sept. 12 essay, Amodei urged the industry to slow the pace of capability improvements to allow safety measures to catch up. He warned that recursive self-improvement, defined as AI’s growing ability to build the next generation of AI, could outrun human understanding if left unchecked.
METR estimated that much of the faster development stems from AI itself helping to build newer versions of the company’s Claude chatbot. The researchers analyzed internal data provided by Anthropic over an 11-week period from mid-August through late October. They also reviewed both public and private risk reports published by the company in August. METR concluded that the public report understated the level of acceleration occurring within the firm.
Sam McCandlish, an Anthropic co-founder and chief architect, responded to the report in a post on X on Thursday. He stated that many of METR’s conclusions match his own assessments. McCandlish wrote that while AI is speeding up research, it is difficult to determine the exact magnitude of the increase. He noted that the company plans to release its own analysis of progress speed within weeks.
McCandlish explained that the collaboration with METR began before the company’s current commitment to embedded third-party evaluators. He called the partnership a good illustration of why such evaluators are beneficial. However, he acknowledged that redactions in the report make it harder for outside observers to verify METR’s work. He described sharing the specific drivers of the speedup as commercially sensitive and a potential roadmap for competitors.
The report includes caveats regarding the scope of the findings. METR noted that the acceleration may be limited to well-specified tasks that can be graded automatically. The group also considered the possibility that Anthropic understated its previous progress. Researchers confidently ruled out increased computing power as the cause of the acceleration. Instead, they attributed the gains to the use of AI in improving AI.
METR found no bottlenecks in computing power or data holding back progress. The report stated that AI is not yet able to make highly significant and fully autonomous contributions to the most important parts of model training. However, Anthropic researchers described signs that this capability is emerging. The nonprofit is funded through donations and has evaluated models for other labs, including OpenAI, for years.
