Featured image: A stock image used by the UK Ministry of Defence to illustrate the front cover of its ‘Human-Machine Teaming’ Joint Concept Note (2018), offering a rather dystopian and somewhat impractical-looking vision of the future of warfare https://bit.ly/36g3ubf The same image is used elsewhere on the internet to advertise TVs.

On 20 November the UK Campaign to Stop Killer Robots wrote to the UK government to engage in discussion on policy in the area of autonomous weapons, and to ask specific questions and for clarifications on aspects of the government’s current position.

In interventions to the Convention on Certain Conventional Weapons – including in a new working paper – the UK has contributed useful content on what human control over weapons systems should entail. Much of this could provide points of common ground that states could build on in international work towards addressing the problem of increasing ‘autonomy’ in weapons systems. The UK has agreed that the subject of ‘human control’ is the key area in which collective work is needed, and its new working paper to the CCW (which it caveats “should not be read as a formal representation of UK policy”) helpfully presents “human dignity and responsibility” as the reasons why human control over weapons is crucial. There is an opportunity for UK government leadership in developing consensus on how meaningful human control can be maintained over weapons systems to uphold these values.

Nevertheless, the Campaign has concerns about several aspects of the current government position, including its promotion of ill-defined ‘humanitarian’ benefits to automating aspects of targeting – as well as the fact that the UK is still against international legal regulation. For the campaign, agreeing a new international treaty in this area is crucial.

Our letter can be read below, or downloaded here.

Update: on 4 January 2021 the Counter Proliferation and Arms Control Centre (a joint government unit of the Ministry of Defence and Foreign Commonwealth and Development Office) sent the campaign a reply on behalf of the UK government. Though encouraging in reiterating that human control should be the focus for international discussion and that in-depth consideration of the issues is needed, it was disappointing that the reply did not address directly what the UK considers the benefits of the technologies under question to be (beyond a general statement about supporting compliance with international humanitarian law), given there are real humanitarian concerns. We look forward to further dialogue on these issues in the future.

 

Rt Hon Ben Wallace MP

Secretary of State

Ministry of Defence

Whitehall

London SW1A 2HB

20 November 2020

Dear Secretary of State,

We are writing on behalf of the UK Campaign to Stop Killer Robots, in advance of the next meeting of the Group of Governmental Experts (GGE) on ‘Lethal Autonomous Weapons Systems’ (LAWS) at the Convention on Certain Conventional Weapons (CCW), as well as the CCW’s meeting of High Contracting Parties. We welcome the UK government’s recognition in the CCW that discussing human control is central to successful international work to address increasing ‘autonomy’ in weapons systems, and that this is an area in which meaningful progress can be made.[1]

In this regard, we encourage the UK government to work on building recognition and stimulating engagement around the valuable content it has already contributed to the CCW on aspects of human control.[2] Such efforts could help promote convergence among states on useful points of substance and common understanding in this area, both at the normative and operational level.

We also welcome the UK’s working paper exploring ‘the human role in autonomous warfare’ and would appreciate the opportunity to discuss its contents further with you and your ministry.[3] We share the paper’s view that ‘assigning responsibility and preserving dignity’ are key reasons for retaining human control over the use of force. However, the UK’s perspective on human control raises concerns that such control may disproportionally focus on the early stages of weapon systems’ research and development. Although we acknowledge the importance of ensuring human-machine interaction throughout a weapon system lifecycle, we believe that further attention should be placed on how to operationally maintain human control over the use of force on actual, specific battlefield decisions, so as to ensure compliance with International Humanitarian and Human Rights Law. As such, we believe that to preserve legal and ethical compliance, meaningful human control requires positive obligations, including to control location and duration of system use, as well as the specification of targets.

In addition, we would appreciate clarification on what ‘humanitarian benefits’ the UK believes could arise from ‘automating some tasks within the targeting process’.[4] We are particularly interested in better understanding what examples or experiences substantiate the UK government’s argument that autonomy could decrease risks for civilians and advance humanitarian goals. We note that the UN Secretary General’s 2020 report on the Protection of Civilians expresses concerns over LAWS. We share the UN Secretary General’s view that developments in weapons technologies could, in fact, present a major challenge to the protection of civilians in armed conflict.[5]

We agree with the UK’s view that substantial contributions towards delineating the principles and components of meaningful human control over weapons systems will be key to building an effective international framework to construct regulation for human control. However, we are not persuaded by the UK’s position that the existing framework is ‘more than sufficient’ to address the novel moral, ethical, human rights and legal issues that developments in this area pose. We also believe that a ‘compendium of good practice’ should be a tool for the discussion of necessary additional rules—rather than an end point. In this regard, we would like to ask what specific examples of ‘good practice’—for instance practical descriptions and case studies regarding the use and parameters of control of current sensor-based weapons systems—the UK might present at the next GGE meeting and beyond, to concretely allow further discussion across the LAWS community of interest.

In light of the above issues,  we noted with interest the comments made by the UK Chief of Defence Staff, General Sir Nick Carter, in an interview with Sky News on 8th November, where he stated that  the UK ‘will absolutely avail ourselves of autonomous platforms and robotics wherever we can’ and that in the near future ‘I suspect we could have an army of 120,000, of which 30,000 might be robots, who  knows’.[6] These remarks appear to be in conflict with the UK’s official stated policy in respect of LAWS, that being that the MoD ‘has no intention to develop systems that operate without human intervention in the weapon command and control chain.’[7] Given this apparent conflict, we would be grateful to learn the extent to which General Sir Nick Carter’s comments reflect a change in UK official policy.

Finally, we are interested to see the UK’s proposal for the greater involvement of industry in international discussions regarding LAWS.[8] Our experience of engaging with the tech and finance industries has impressed on us the fact that many key stakeholders would welcome clearer international legal regulation to protect their work, ensuring it will not be used for dangerous or unethical purposes and/or safeguarding it against reverse engineering for such unintended applications. We believe the contribution of these industries to international discussions would be welcome, and would be interested to know which sectors and stakeholders the UK expects to invite to help form policy in this area.

We look forward to hearing your response and more detail about the approach the UK government will be taking as the conversation continues in the lead-up to the critical moment of the next CCW Review Conference (currently scheduled for 2021). Ultimately, we believe that the UK can contribute to the success of the overall process by demonstrating leadership in working with other states to both develop a strong shared consensus of the practicalities of meaningful human control over weapons systems and the use of force, as well as building understanding on how to preserve responsibility and human dignity. From the campaign’s perspective, we will continue to work with states to prepare for formal negotiations on a legal framework that prohibits and restricts lethal autonomous weapon systems.

Yours sincerely,

Ann Feltham

Parliamentary Co-ordinator, Campaign Against Arms Trade (CAAT)

Ben Donaldson

Head of Campaigns, United Nations Association UK (UNA-UK)

Chris Cole

Director, Drone Wars UK

Dave Webb

Chair, Campaign for Nuclear Disarmament (CND)

Oliver Feeley-Sprague

Programme Director, Military, Security and Police, Amnesty International UK

Maiara Folly,

Coordinator, UK Campaign to Stop Killer Robots

Dr Rebecca E. Johnson

Director, Acronym Institute for Disarmament Diplomacy

Richard Moyes

Managing Director, Article 36

Robert Parker

Director of Policy and Communications, Saferworld

Dr Stuart Parkinson

Executive Director, Scientists for Global Responsibility (SGR)

Taniel Yusef

International Representative, The Women’s International League for Peace and Freedom UK (WILPF UK)

Amnesty International student representatives at the universities of Birmingham, Bristol, Edinburgh, Newcastle, Oxford and Warwick.

 

[1] UK commentary on the Guiding Principles: https://documents.unoda.org/wp-content/uploads/2020/09/20200901-United-Kingdom.pdf

[2] For our analysis of UK contributions up to 2020, see Article 36, ‘From “pink eyed terminators” to a clear-eyed policy response? UK government policy on autonomy in weapons systems’ https://article36.org/wp-content/uploads/2020/03/UK-policy-on-autonomy-in-weapons-systems-2020.pdf

[3] UK Expert paper: The human role in autonomous warfare https://undocs.org/Home/Mobile?FinalSymbol=CCW%2FGGE.1%2F2020%2FWP.6&Language=E&DeviceType=Desktop

[4] UK Expert paper: The human role in autonomous warfare

[5] UN Secretary General 2020 report on the Protection of Civilians in Armed Conflicts https://reliefweb.int/sites/reliefweb.int/files/resources/S_2020_366_E.pdf

[6] Sky News, “World War Three ‘a risk’, says UK defence chief”, https://www.youtube.com/watch?v=ACDlPOssea0&feature=emb_logo

[7] Ministry of Defence, Joint Doctrine Publication 0-30.2: Unmanned Aircraft Systems:

https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/673940/doctrine_uk_uas _jdp_0_30_2.pdf

[8] UK commentary on the Guiding Principles: https://documents.unoda.org/wp-content/uploads/2020/09/20200901-United-Kingdom.pdf