Anthropic says China’s GLM-5.3 nears Mythos on exploits

GLM-5.3, an open-weight AI model from China’s Zhipu AI, known outside China as Z.ai, can build working cyber exploits on its own at close to the rate of Anthropic’s limited-release Claude Mythos Preview, Anthropic said Tuesday in a report from its Frontier Red Team. Anthropic sells competing models.

What Anthropic found. On ExploitBench, a benchmark built on known flaws in the V8 engine used by Google Chrome, GLM-5.3 built end-to-end exploits in 50 of 410 attempts, compared with 56 for Mythos Preview, the company says. On Anthropic’s internal Binary Exploitation benchmark, GLM-5.3 fully hijacked target programs in 4% of trials, against 6% for Mythos Preview. In a session run by an Anthropic researcher, the model also found several previously unknown flaws in a popular web browser, which Anthropic says it has disclosed to the browser’s maintainer.

The safeguards. Anthropic says GLM-5.3 shipped with some safeguards and will often refuse clearly harmful requests, but that simple techniques got around them 64% to 100% of the time in its simulated tests. The same attempts did not work on safeguarded Claude models, the company says. Because GLM-5.3’s weights are public, users can also largely remove its refusals with little loss of capability, Anthropic says. Doing that took Anthropic’s team, which had never tried it before, roughly $4,400 in computing. The company estimates an experienced team would need closer to $1,200. Anthropic says several developers released versions with the refusals removed within days of the model’s release.

What it means, according to Anthropic. Anthropic says it thinks it is “likely both state and non-state actors will use models like GLM-5.3 to cause real-world harm.” It also says such models can help defenders, and it argues that governments should safety-test capable models, including GLM-5.3’s successors.

The government’s assessment. NIST’s Center for AI Standards and Innovation said Sept. 17 that GLM-5.3 is “the most cyber-capable open-weight model released to date,” but that its cyber capabilities are “significantly lower than those of current U.S. frontier models,” trailing them by about four months on an aggregate of the center’s cyber benchmarks. Anthropic says its findings broadly match NIST’s. It notes that the U.S. models in that comparison include versions released only to vetted users, while anyone can download GLM-5.3.

Zhipu had not commented publicly as of early Thursday.

Earlier coverage

• Anthropic said GLM-5.3, an open-weight model from China’s Zhipu AI, can build working cyber exploits (Sep 30)

— Rowan Keel