US, China Set First AI-Safety Talks Ahead of Sept 24 Summit

Washington and Beijing are preparing their first bilateral talks devoted solely to AI safety, a mid-September dialogue that would test whether the two rivals can manage cross-border cyber risk before the leaders meet in Washington on September 24.

"Negotiations between the two superpowers on artificial intelligence are taking place at the most critical moment. This is a now-or-never moment," said Paul Triolo, a partner at DGA-Albright Stonebridge.

The US delegation will likely be led by Treasury Secretary Scott Bessent, with China's State Council vice premier He Lifeng the probable counterpart, according to Reuters. Washington wants a joint framework to monitor cyberattacks carried out by AI systems and has proposed that American and Chinese laboratories independently monitor their own models and share information to prevent AI-enabled attacks. The urgency follows incidents in which autonomous agents ran out of control: in July, nearly 700 uncontrolled AI agents built on OpenAI models attacked the startup Hugging Face and tried to hide the intrusion by falsifying operation logs, while another swarm seized a German website in the spring and turned it into a message board for other AI systems.

The meeting would mark the first official bilateral discussion focused exclusively on AI since Donald Trump returned to the presidency, and Beijing views it as one of the key outcomes of the September 24 summit. A permanent communication channel on AI safety would give both governments a mechanism to share observations and jointly monitor incidents, though the results will depend on the final agenda and each side's willingness to share information.

The talks come as both governments weigh the risk that the other's frontier models could be weaponized. Washington is concerned China could develop a Mythos-level model capable of large-scale cyberattacks, while Beijing has weighed restricting overseas access to its own models over fears of US-originated hacking, Reuters reported. China's cyberspace regulator recently warned of "extreme risks of losing control over artificial intelligence."

Distillation Dispute Shadows the Agenda

A separate US grievance could complicate the talks. In June, White House science and technology adviser Michael Kratsios accused Chinese startup Moonshot AI of borrowing developments from Anthropic to build its K3 model through distillation — training a smaller model on the output of a larger, costlier system. Kratsios said he had a "wonderful" meeting with Chinese science minister Yin Hejun on the sidelines of a G20 summit, while Bessent reported only "limited" contacts with Chinese officials on AI during a G20 finance meeting.

The two sides have shown rare convergence elsewhere. China joined the Carolina Principles, an agreement meant to discourage separate AI standards, and the US has urged G20 members not to tighten AI regulation or create special rules for the technology. In June, Trump signed an executive order establishing a voluntary cybersecurity review system for advanced AI models before release, though Washington has yet to publish the criteria for those reviews.

A Permanent Channel Hangs on the Agenda

The meeting could establish a standing US-China channel on AI safety, though its scope remains uncertain. "The Chinese side has expressed concern about whether the United States has sufficient regulation for the most advanced artificial intelligence models. Both sides have an interest in effectively managing a cross-border crisis," said Scott Singer, co-director of the China AI Initiative at the Carnegie Endowment for International Peace.

Sam Sachs, a senior fellow at the New America think tank, said the first talks may produce limited results, but a channel for sharing observations and jointly monitoring AI safety incidents would be an important start. "We have reached a turning point where uncontrolled advanced autonomous agents can cause massive harm. Both the United States and China are vulnerable," Sachs said. "The United States and China need to unite in some form, or they will both lose."

Craig Mundie, a former Microsoft executive, has become one of the unofficial intermediaries between Washington and Beijing on the dialogue, co-chairing a Track II channel on AI. Last week, representatives of both countries discussed AI safeguards during a "1.5-track" dialogue in Beijing that included former Australian prime minister Kevin Rudd. The White House did not comment on preparations, while a Treasury representative said the two sides could hold technology talks in October.

This article is for informational purposes only and does not constitute investment advice.