Related: Ideas for AI labs: Reading list. See also: AI labs' statements on governance.

This document is about AI policy ideas. It's largely from an x-risk perspective. Strikethrough denotes sources that I expect are less useful for you to read.

Lists

Lists of government (especially US government) AI policy ideas. I recommend carefully reading the lists in the first ~5 bullets in this list, noticing ideas to zoom in on, and skipping the rest of this section.

• Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims (Brundage et al. 2020)
  • Followed up by Filling gaps in trustworthy development of AI (Avin et al. 2021)
• 12 tentative ideas for US AI policy (Muehlhauser 2023) (EAF)
• Survey on intermediate goals in AI governance (Räuker and Aird 2023)
• Policymaking in the Pause (FLI 2023) (LW)
• Frontier AI Regulation: Managing Emerging Risks to Public Safety (Anderljung et al. 2023)
• "30 actions to reduce existential risk" in Existential risk and rapid technological change (Stauffer et al. 2023)
• The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation (Brundage et al. 2018)
• How major governments can help with the most important century (Karnofsky 2023) (LW)
• [Chuck Schumer] (2023)
• [Dan Hendrycks] (2023)
• "Discussion" in "Verifying Rules on Large-Scale NN Training via Compute Monitoring" (Shavit 2023)
• Future Proof (CLTR 2021)
• Year 1 Report (National Artificial Intelligence Advisory Committee 2023)
• Final Report (National Security Commission on Artificial Intelligence 2021)
• Challenges to U.S. National Security and Competitiveness Posed by AI (Matheny 2023)
• "Policy Options" in "Artificial Intelligence and Strategic Trade Controls" (Viski et al. 2020)
• Existential and global catastrophic risk policy ideas database (filter for "Artificial intelligence") (Sepasspour et al. 2022)
• Various private lists and works in progress

Levers

Some sources focus on policy levers rather than particular policy proposals.

• "Governmental levers" in "Literature Review of Transformative AI Governance" (Maas draft)
  • This report is excellent
• AI Policy Levers: A Review of the U.S. Government's Tools to Shape AI Research, Development, and Deployment (Fischer et al. 2021)
• "Affordances" in "Framing AI strategy" (Stein-Perlman 2023)
• Current UK government levers on AI development (Hadshar 2023)
• Standards (largely a non-government lever)
  • How technical safety standards could promote TAI safety (O'Keefe et al. 2022)
  • Standards for AI Governance (Cihon 2019)
  • Actionable-guidance and roadmap recommendations for the NIST AI Risk Management Framework (Barrett et al. 2022) (LW)
  • AI Risk Management Framework (NIST 2023)
  • Ethically Aligned Design (IEEE 2017)
  • Global AI Standards Repository (OCEANIS)

Other policy guidance

• Five considerations to guide the regulation of "General Purpose AI" in the EU's AI Act (AI Now Institute 2023)
• AI Accountability Policy (National Telecommunications and Information Administration 2023)

Desiderata

Some sources focus on abstract desiderata rather than how to achieve them.

• Asilomar AI Principles (FLI 2017)
• Blueprint for an AI Bill of Rights (OSTP 2022)
• OECD AI Principles (OECD 2019)
  • I think there are other relevant OECD reports or recommendations
• Policy Desiderata for Superintelligent AI (Bostrom et al. 2018)
• Universal Guidelines for Artificial Intelligence (The Public Voice 2018)
• AI Policy Challenges (FLI 2018)
• Artificial Intelligence: the global landscape of ethics guidelines (Jobin et al. 2019)

Ideas^[1]

• Verifying Rules on Large-Scale NN Training via Compute Monitoring (Shavit 2023) (LW)
• Why and How Governments Should Monitor AI Development (Whittlestone and Clark 2021)
• Regulatory Markets: The Future of AI Governance (Hadfield and Clark 2023)
• Why we need a new agency to regulate advanced artificial intelligence: Lessons on AI control from the Facebook Files (Korinek 2021)
• Compute Funds and Pre-trained Models (Anderljung 2022)
• AI & Antitrust: Reconciling Tensions Between Competition Law and Cooperative AI Development (Hua and Belfield 2021)
  • See also Antitrust-compliant AI industry self-regulation (O'Keefe 2021)
  • See also AI & antitrust/competition law (Aird 2022)
• How We Can Regulate AI (Balwit 2023) [compute governance]
• Regulating artificial intelligence: Proposal for a global solution (Erdélyi and Goldsmith 2022)
• Immigration Policy and the Global Competition for AI Talent (Huang and Arnold 2020)
• AI & Global Governance: Why We Need an Intergovernmental Panel for Artificial Intelligence (Miailhe 2018)
• Liability
  • New: LPP on liability
  • Products liability law as a way to address AI harms (Villasenor 2019)
  • Liability for artificial intelligence and other emerging digital technologies (European Commission 2019).
  • Footnote 16 of Collective action on artificial intelligence: A primer and review (de Neufville and Baum 2021) points to academic work on AI and liability
• Crafting Legislation to Prevent AI-Based Extinction (Cohen and Osborne 2023)
  • This is technically a response to government request for comment, but is really a policy proposal
• An AI Policy Tool for Today: Ambitiously Invest in NIST (Anthropic 2023)
  • Or see the policy memo version of this blogpost
• Sharing Powerful AI Models (Shevlane 2022)
• You Can't Regulate What You Don't Understand (O'Rielly 2023)
• Killer Robots Are Here—and We Need to Regulate Them (Trager and Luca 2022)
• Incident reporting & tracking
  • See PAI's AI Incidents Database (arXiv, blogpost)

Policy proposals in the mass media

• AI's Gatekeepers Aren’t Prepared for What's Coming (FP: Scharre 2023)
  • This piece is great
• The world needs an international agency for artificial intelligence, say two AI experts (Economist: Marcus and Reuel 2023)
• We Need a Manhattan Project for AI Safety (POLITICO: Hammond 2023)
• Does the world need an arms control treaty for AI? (CyberScoop: Groll 2023)
• AI Desperately Needs Global Oversight (WIRED: Chowdhury 2023)
• The Surprising Thing A.I. Engineers Will Tell You if You Let Them (NYT: Klein 2023)
• We Must Regulate A.I. Here’s How. (NYT: Khan 2023)
• We Must Declare Jihad Against A.I. (Compact: Cuenco 2023)

Responses to government requests for comment

• Response to the NTIA AI Accountability Policy (GovAI 2023)
• GovAI Response to the Future of Compute Review - Call for Evidence (GovAI 2022)
• Future of compute review - submission of evidence (CLTR et al. 2022)
• Anthropic Comment Regarding "Study To Advance a More Productive Tech Economy" (Anthropic 2022)
• National Security Addition to the NIST AI RMF (Special Competitive Studies Project 2023)
• Reconfiguring Resilience for Existential Risk (CSER 2021)
• Response to the UK's Future of Compute Review (CLTR et al. 2023)
• Submission to the NIST AI Risk Management Framework (GovAI 2022)
• Submission of Feedback to the European Commission's Proposal for a Regulation laying down harmonised rules on artificial intelligence (CSER 2021)
• Advice to UN High-level Panel on Digital Cooperation (CSER and GovAI 2019)
• Consultation on the European Commission's White Paper on Artificial Intelligence: a European approach to excellence and trust (GovAI 2020)

See also

Very non-exhaustive.

• AI Governance: A Research Agenda (Dafoe 2018)
• Slowing AI (Stein-Perlman 2023)
• Survey on intermediate goals in AI governance (Räuker and Aird 2023)
• FACT SHEET: Biden-⁠Harris Administration Announces New Actions to Promote Responsible AI Innovation that Protects Americans’ Rights and Safety (The White House 2023)
• OECD AI Policy Observatory
• PAI and the AI and Shared Prosperity Initiative
• [Mark Warner] (2023)
• EU AI Act (EU)
• CHIPS Act (US)
• A pro-innovation approach to AI regulation (UK Office for Artificial Intelligence 2023)

This post is largely missing policy levers beyond domestic policy: international relations,^[2] international organizations, and standards.

Some sources are roughly sorted within sections by a combination of x-risk-relevance, quality, and influentialness– but sometimes I didn't bother to try to sort them, and I've haven't read all of them.

Please have a low bar to suggest additions, substitutions, rearrangements, etc.

Thanks to Jakub Kraus and Seán Ó hÉigeartaigh for some sources.

Last updated: 10 July 2023.

1. ^{^}

   - I think these sources each focus on a particular policy idea– I haven't even skimmed all of them
   - Very non-exhaustive
   - Thanks to Sepasspour et al. 2022 and a private list for some sources

2. ^{^}

   International agreements seem particularly important and neglected.

   One source (not focused on policy ideas or levers): Nuclear Arms Control Verification and Lessons for AI Treaties (Baker 2023).

   Oliver Guest agrees that there are not amazing sources but mentions:

   - CSET on international security in the context of military AI
   - CNAS on international arms control and confidence-building measures for military AI