Research and News
The below database includes IAPS’ public research reports, responses to government requests for information, blog posts, and more.
A look at U.S. and Chinese policy landscapes reveals differences in how the two countries approach the governance of general-purpose artificial intelligence. Three areas of divergence are notable for policymakers: the focus of domestic AI regulation, key principles of domestic AI regulation, and approaches to implementing international AI governance.
We are excited to announce that Jenny Marron has joined IAPS as our new Director of Policy and Engagement.
In this commentary, we explore key questions for the International Network of AI Safety Institutes and suggest ways forward given the upcoming San Francisco convening on November 20-21, 2024. What should the network work on? How should it be structured in terms of membership and central coordination? How should it fit into the international governance landscape?
Based on a systematic review of open sources, we identify Chinese “AISI counterparts,” i.e. Chinese institutions doing similar work to the US and UK AISIs and that have relatively close government links.
This post is a copy of IAPS’ response to a BIS request for public comment. It outlines ways to expand the role of other stakeholders in the reporting process for AI models and compute clusters, including third-party evaluators, civil society groups, and other public sector entities.
AI Safety Institutes (AISIs) are a new institutional model for AI governance that has expanded across the globe. In this primer, we analyze the “first wave” of AISIs: the shared fundamental characteristics and functions of the institutions established by the UK, the US, and Japan that are governmental, technical, with a clear mandate to govern the safety of advanced AI systems.
IAPS submitted a response to a National Institute of Standards and Technology (NIST) Request for Comment, outlining practices that could help AI developers better manage and mitigate misuse risks from dual-use foundation models.
IAPS submitted a response to a Department of Defense Request for Information on Defense Industrial Base Adoption of AI for Defense Applications.
This report analyzes the research published by Anthropic, Google DeepMind, and OpenAI about safe AI development, as well as corporate incentives to research different areas. This research reveals where corporate attention is concentrated and where there are potential gaps.
This piece is a link post for a paper which was led by Hadrien Pouget (Carnegie Endowment for International Peace) and Claire Dennis (Centre for the Governance of AI). IAPS staff Renan Araujo and Oliver Guest were among the paper’s co-authors.
This policy memo by Jam Kraprayoon and Bill Anderson-Samways, published by the UKDayOne and the Social Market Foundation, recommends that the UK government implement a targeted market-shaping program mobilizing public and private sector investment to supercharge the UK’s AI assurance technology industry.
This policy memo by Jam Kraprayoon, Joe O’Brien, and Shaun Ee (IAPS), published by the Federation of American Scientists, proposes that Congress should set up an early warning system for novel AI-enabled threats to provide defenders maximal time to respond to a given capability before information about it is disclosed or leaked to the public.
Future AI systems may be capable of enabling offensive cyber operations, lowering the barrier to entry for designing and synthesizing bioweapons, and other high-consequence dual-use applications. If and when these capabilities are discovered, who should know first, and how? We describe a process for information-sharing on dual-use capabilities and make recommendations for governments and industry to develop this process.
This report analyzes the potential impact of high-end consumer GPUs on the efficacy of US export controls on AI chips. It studies three stockpiling scenarios and what AI capabilities those may enable, and makes recommendations for policymakers.
Adding location verification features to AI chips could unlock new governance mechanisms for regulators, help enforce existing and future export controls by deterring and catching smuggling attempts, and enable post-sale verification of chip locations. This paper is meant to serve as an initial introduction to location verification use-cases for AI chips with comparison of different methods.
This blog post by Erich Grunewald (IAPS) and Samuel Hammond (the Foundation for American Innovation) argues that Congress should increase the funding of the Bureau of Industry and Security.
This issue brief suggests agenda items for dialogues about advanced AI risks that minimize risk of leaking sensitive information.
This issue brief analyzes key AI-related allocations from the Biden FY2025 Presidential Budget in terms of their potential impact on the responsible development of advanced AI.
Mitigating the risks from frontier AI systems requires up-to-date and reliable information about those systems. Organizations that develop and deploy frontier systems have significant access to such information. By reporting safety-critical information to actors in government, industry, and civil society, these organizations could improve visibility into new and emerging risks posed by frontier systems.
A systematic search for potential case studies relevant to advanced AI regulation in the United States, looking at all federal agencies for factors such as level of expertise, use of risk assessment, and analysis of uncertain phenomena.
This issue brief evaluates the original example of a Responsible Scaling Policy (RSP) – that of Anthropic – against guidance on responsible capability scaling from the UK Department for Science, Innovation and Technology (DSIT).
On this episode of the Federal Drive with Tom Temin, IAPS consultant Onni Aarne discusses how specialized AI chips, and the systems that use them, need protection from theft and misuse. The podcast episode and interview transcript are available on the Federal News Network.
IAPS’s response to a NIST RFI, outlining specific guidelines and practices that could help AI actors better manage and mitigate risks from AI systems, particularly from dual-use foundation models.
Today, the Center for a New American Security (CNAS), in collaboration with the Institute for AI Policy and Strategy, has released a new report, Secure, Governable Chips, by Onni Aarne, Tim Fist, and Caleb Withers.
The report introduces the concept of “on-chip governance,” detailing how security features on AI chips could help mitigate national security risks from the development of broadly capable dual-use AI systems, while protecting user privacy.
This paper examines the Federal Select Agent Program, the linchpin of US biosecurity regulations. It then draws out lessons for AI regulation regarding licensing, regulatory expertise, and the merits of “risk-based” vs. “list-based” systems.
This primer introduces the topic of Chinese AI chip making, relevant to understanding and forecasting China's progress in producing AI chips indigenously.
With this paper, we aim to help actors who support alignment efforts to make these efforts as effective as possible, and to avoid potential adverse effects.
This paper discusses how external scrutiny (such as third-party auditing, red-teaming, and researcher access) can bring public accountability to bear on decisions regarding the development and deployment of frontier AI models.
We link to a working paper which was led by Tim Fist of the Center for a New American Security, and coauthored with IAPS researcher Erich Grunewald. It builds on IAPS's earlier report on AI chip smuggling into China.
Events that bring together international stakeholders to discuss AI safety are a promising way to reduce AI risks. This report recommends ways to make these events a success.