The independent resource on global security

Limits on Autonomy in Weapon Systems: Identifying Practical Elements of Human Control

Front cover SIPRI ICRC_Limits of Autonomy
Publisher: SIPRI
SIPRI, Stockholm:
June, 2020

There is wide recognition that the need to preserve human control over weapon systems and the use of force in armed conflict will require limits on autonomous weapon systems (AWS).

This report from the Stockholm International Peace Research Institute and the International Committee of the Red Cross offers in-depth analysis of the type and degree of human control that is required to mitigate the risks posed by AWS. It proposes three types of control measures to reduce or compensate for the unpredictability introduced by AWS and associated risks for civilians: controls on the weapon’s parameters such as types of targets, controls on the environment of use and controls in the form of human supervision.

Limits on Autonomy in Weapon Systems: Identifying Practical Elements of Human Control is a comprehensive examination of the specific controls on AWS needed to ensure human control over the use force, and to address legal, ethical and operational concerns. It provides policymakers with practical guidance on how these control measures should form the basis of internationally agreed limits on AWS—whether rules, standards or best practices.
 

Contents

1. Introduction

2. The exercise of human control from legal, ethical and operational perspectives

3. Operationalizing human control

4. Key findings and recommendations

ABOUT THE AUTHOR(S)/EDITORS

Dr Vincent Boulanin is Director of the Governance of Artificial Intelligence Programme at SIPRI.
Moa Peldán Carlsson was a Research Assistant on emerging military and security technologies.
Netta Goussac was an Associate Senior Researcher within SIPRI’s Armament and Disarmament research area.
Neil Davison is a senior adviser in the Department of International Law and Policy at the ICRC’s headquarters in Geneva.