Chinese AI Safety Institute Counterparts

In late 2023, the US and UK established AI Safety Institutes (AISIs). They were followed by various other jurisdictions but not, to date, by China. Based on a systematic review of open sources, we identify Chinese “AISI counterparts,” i.e. government-linked Chinese institutions doing similar work to the US and UK AISIs.

To the extent that AISIs and other bodies seek to engage with Chinese counterparts, the specific counterparts in Table 1 appear to be most promising. We discuss additional potential counterparts in the body of the paper.

Table 1: Most promising Chinese AI Safety Institute counterparts		AI Safety Institute core functions
Institution	Recommended topics	Technical research and evaluations	Standards	International cooperation
CAICT, a think tank housed within the Ministry of Industry and Information Technology.	Evaluations	✓	✓	✓
Shanghai AI Lab, a government-backed AI research institution.	Technical research and evaluations and international cooperation	✓	✓	✓
TC260, a committee within China’s official national standards body.	Standards		✓
Institute for AI International Governance, a policy-focused research institute within Tsinghua University.	International cooperation			✓
Beijing Academy of Artificial Intelligence, a government-backed AI research institution.	International cooperation	✓	✓	✓

In the rest of the executive summary, we provide more information about these five institutions. We group them by the “core AISI functions” that US and UK AISI collectively perform. We provide a comprehensive summary, covering every institution described in the paper, below.

Technical research

The US and UK AISI both perform safety evaluations on some AI systems. Such work is included within our “technical research” category.

CAICT (China Academy for Information and Communications Technology) is a think tank housed within the Ministry of Industry and Information Technology.

• CAICT performs AI evaluations via its “Fangsheng” platform. This assesses AI outputs for various aspects of “safety”, including gender bias, “public order and morality,” and violent content. A CAICT report about Fangsheng prominently cited Geoffrey Hinton’s concerns about AI “taking over” humanity. A benchmark included in Fangsheng includes some elements relevant to these concerns, such as apparently testing for “appeals for rights” in AI outputs.

• A paper from CAICT about “large [AI] model governance” discusses various possible risks from such models. These include sexist stereotypes being reproduced, AI-assisted cyberattacks, and humans losing control over AI systems.

• CAICT likely has a high degree of influence over the AI industry in China via its leadership role in China’s Artificial Intelligence Industry Alliance (AIIA), an industry grouping.

Shanghai AI Lab is a government-backed research institution. It primarily aims to support the Chinese AI industry and contribute technical AI breakthroughs. Although safety is not its stated focus, it has done several pieces of work that are highly relevant to AISIs’ focuses. Key examples include:

• OpenCompass is a widely used AI evaluations platform. It includes some safety benchmarks from other groups, such as TruthfulQA (testing the truthfulness of LLMs) and Adversarial GLUE (measuring LLMs’ robustness to adversarial attacks).

• SALAD-Bench is a safety benchmark covering risks across various categories. Risks in scope include generating toxic content, assisting users with biological, chemical, and cyber weapons, as well as “persuasion and manipulation”. The Lab has also published “FLAMES,” a benchmark for value alignment.

Safety standards

TC260 (National Cybersecurity Standardization Technical Committee 260) is a committee within China’s official national standards body. Its work covers a broad range of technology topics, so engagement would ideally focus on select individuals who have been directly involved in AI safety standards. Key examples of TC260 work on AI safety include:

• A voluntary AI risk management framework. The document discusses risks including bias, misinformation, cybersecurity issues, lowering barriers to biological and chemical weapons, and loss of human control over advanced AI systems. 

A technical standard for testing the safety/security of generative AI outputs. The testing processes included testing for bias, privacy violations, and political control over generated content. An initial draft of the document also mentioned “long-term risks” from AI, including AI deception and biological weapons production. The technical standard is being adapted into a more authoritative national standard, the first draft of which did not refer to long-term risks.

Two other standards groups, TC28/SC42 and CESA, might be very relevant counterparts in the future, though they have not yet done enough work overlapping with US and UK AISI for us to strongly recommend them as counterparts.

International cooperation

I-AIIG (Institute for AI International Governance) is a research institute within Tsinghua University focusing on policy research about (international) AI governance. The institute’s leadership has repeatedly spoken about being concerned about extreme AI risks. Activities from I-AIIG to promote international cooperation on AI safety and governance include:

• Organizing the International Forum on AI Cooperation and Governance for Chinese and non-Chinese experts. The most recent Forum included a sub-event focusing on the safety of advanced AI.

• Participating in various track II diplomacy events relating to AI.

Beijing Academy of Artificial Intelligence (BAAI) is a government-backed research institute doing cutting-edge AI development. Senior figures at BAAI (such as HUANG Tiejun and ZHANG Hongjiang) have expressed concern about extreme risks from AI and there is some technical work from the organization on this topic. That said, BAAI’s most significant contributions to AI safety are likely its work on promoting international cooperation about the topic:

• BAAI’s past two yearly conferences have included an AI safety “forum.” This has included talks about AI safety topics from Chinese scientists about AI safety, as well as non-Chinese experts such as Stuart Russell and Victoria Krakovna.

• BAAI is closely involved with the International Dialogue on AI Safety (IDAIS), a series of gatherings between AI experts, primarily from China and Western countries. The three dialogues that have occurred so far have led to joint statements expressing strong concern about AI risks and calling for international cooperation to reduce them. 

Shanghai AI Lab, described above for its work on technical research and safety evaluations, also intends to increase its international engagement efforts. However, there are limited examples of that effort so far.