Chinese AI Safety Institute Counterparts
Based on a systematic review of open sources, we identify Chinese “AISI counterparts,” i.e. Chinese institutions doing similar work to the US and UK AISIs and that have relatively close government links.
Coordinated Disclosure of Dual-Use Capabilities: An Early Warning System for Advanced AI
Future AI systems may be capable of enabling offensive cyber operations, lowering the barrier to entry for designing and synthesizing bioweapons, and other high-consequence dual-use applications. If and when these capabilities are discovered, who should know first, and how? We describe a process for information-sharing on dual-use capabilities and make recommendations for governments and industry to develop this process.
AI-Relevant Regulatory Precedents: A Systematic Search Across All Federal Agencies
A systematic search for potential case studies relevant to advanced AI regulation in the United States, looking at all federal agencies for factors such as level of expertise, use of risk assessment, and analysis of uncertain phenomena.
Responsible Scaling: Comparing Government Guidance and Company Policy
This issue brief evaluates the original example of a Responsible Scaling Policy (RSP) – that of Anthropic – against guidance on responsible capability scaling from the UK Department for Science, Innovation and Technology (DSIT).
Federal Drive with Tom Temin podcast interview: Onni Aarne on AI hardware security risks
On this episode of the Federal Drive with Tom Temin, IAPS consultant Onni Aarne discusses how specialized AI chips, and the systems that use them, need protection from theft and misuse. The podcast episode and interview transcript are available on the Federal News Network.
Response to the NIST RFI on Auditing, Evaluating, and Red-Teaming AI Systems
IAPS’s response to a NIST RFI, outlining specific guidelines and practices that could help AI actors better manage and mitigate risks from AI systems, particularly from dual-use foundation models.
Secure, Governable Chips
Today, the Center for a New American Security (CNAS), in collaboration with the Institute for AI Policy and Strategy, has released a new report, Secure, Governable Chips, by Onni Aarne, Tim Fist, and Caleb Withers.
The report introduces the concept of “on-chip governance,” detailing how security features on AI chips could help mitigate national security risks from the development of broadly capable dual-use AI systems, while protecting user privacy.
Catching Bugs: The Federal Select Agent Program and Lessons for AI Regulation
This paper examines the Federal Select Agent Program, the linchpin of US biosecurity regulations. It then draws out lessons for AI regulation regarding licensing, regulatory expertise, and the merits of “risk-based” vs. “list-based” systems.
Introduction to AI Chip Making in China
This primer introduces the topic of Chinese AI chip making, relevant to understanding and forecasting China's progress in producing AI chips indigenously.
Safeguarding the Safeguards: How Best to Promote Alignment in the Public Interest
With this paper, we aim to help actors who support alignment efforts to make these efforts as effective as possible, and to avoid potential adverse effects.
Towards Publicly Accountable Frontier LLMs: Building an External Scrutiny Ecosystem under the ASPIRE Framework
This paper discusses how external scrutiny (such as third-party auditing, red-teaming, and researcher access) can bring public accountability to bear on decisions regarding the development and deployment of frontier AI models.
Preventing AI Chip Smuggling to China
We link to a working paper which was led by Tim Fist of the Center for a New American Security, and coauthored with IAPS researcher Erich Grunewald. It builds on IAPS's earlier report on AI chip smuggling into China.
International AI Safety Dialogues: Benefits, Risks, and Best Practices
Events that bring together international stakeholders to discuss AI safety are a promising way to reduce AI risks. This report recommends ways to make these events a success.
Managing AI Risks in an Era of Rapid Progress
This paper discusses risks from future AI systems and proposes priorities for AI R&D and governance. Its many authors include an IAPS researcher, Turing Prize winners, and a Nobel Memorial Prize winner.
Adapting Cybersecurity Frameworks to Manage Frontier AI Risks: a Defense-in-Depth Approach
The complex and evolving threat landscape of frontier AI development requires a multi-layered approach to risk management (“defense-in-depth”). By reviewing cybersecurity and AI frameworks, we outline three approaches that can help identify gaps in the management of AI-related risks.
AI Chip Smuggling into China: Potential Paths, Quantities, and Countermeasures
This report examines the prospect of large-scale smuggling of AI chips into China and proposes six interventions for mitigating that.
Open-Sourcing Highly Capable Foundation Models
This paper, led by the Centre for the Governance of AI, evaluates the risks and benefits of open-sourcing, as well as alternative methods for pursuing open-source objectives.
Deployment Corrections: An Incident Response Framework for Frontier AI Models
This report describes a toolkit that frontier AI developers can use to respond to risks discovered after deployment of a model. We also provide a framework for AI developers to prepare and implement this toolkit.