Understanding the First Wave of AI Safety Institutes: Characteristics, Functions, and Challenges
In November 2023, the UK and US announced the creation of their AI Safety Institutes (AISIs). Five other jurisdictions have followed in establishing AISIs or similar institutions, with more likely to follow. While there is considerable variation between these institutions, there are also key similarities worth identifying.
This report primarily describes one cluster of similar AISIs, the “first wave,” consisting of the Japan, UK, and US AISIs. Additionally, we compare the first wave to other AISI-like institutions in the EU, Canada, France, and Singapore.
First-wave AISIs have several fundamental characteristics in common:
Technical government institutions.
Clear mandate related to the safety of advanced AI systems. First-wave AISIs do not have “catch-all” responsibilities for AI within a jurisdiction.
No regulatory powers.
Safety evaluations are at the center of first-wave AISIs. These are techniques that test AI systems across tasks to understand their behavior and capabilities on relevant risks, such as cyber, chemical, and biological misuse.
First-wave AISIs have three core functions: research, standards, and cooperation. These functions are critical to AISIs’ work on safety evaluations, but also support other activities such as scientific consensus-building and foundational AI safety research:
Research: AISIs so far have emphasized research aimed at advancing the “science of AI safety” through technical work that is mainly empirical and problem-oriented, especially in the case of the UK and US AISIs. This has included research and testing around evaluations, foundational AI safety research, and efforts to build the field of AI safety as a whole.
Standards are a more prescriptive function that AISIs play where they directly influence the practices of industry, government, and society through guidelines and protocols. AISIs’ work on standards ranges from light-touch development of standardized procedures, such as in Japan and the US, to directly setting standards that feed into a wider regulatory process, such as in the EU.
Cooperation is the role that AISIs play as a bridge between governments, industry, and society. AISIs also leverage their technical focus and state-backed legitimacy to boost international coordination on AI safety through initiatives like convenings and scientific consensus-building.
However, despite its growing popularity as an institutional reference, the AISI model is not free from challenges and limitations. Some analysts have criticized the first wave of AISIs for:
Specializing too much on a sub-area and potentially neglecting concerns related to fields like national competitiveness and innovation, or fairness and bias.
Potential redundancies with existing institutions, such as existing standards-developing bodies.
Their relationship with industry, which has been productively close but might affect their impartiality.
Future developments may rapidly change this landscape, and particularities of individual AISIs may not be captured by our broad-strokes description. This policy brief aims to outline the core elements of first-wave AISIs as a way of encouraging and improving conversations on this novel institutional model, acknowledging this is just a simplified snapshot rather than a timeless prescription.
