The following statement is an initial reaction from members of the UK Stop Killer Robots campaign

As part of the long-overdue Defence Artificial Intelligence Strategy, the UK Government has unveiled a new policy on autonomous weapons – so-called “killer robots”.

Published as an annex on “Lethal Autonomous Weapons Systems”, to an accompanying policy paper, the Government recognises that systems which identify, select and attack targets without “context-appropriate human involvement” would be unacceptable.

This is significant – as it continues to recognise that there is a line that should not be crossed in terms of machines making decisions in combat. It is also positive that the new UK position recognises that ethical principles, along with the law, are important for determining where this line should be drawn.  We also welcome that the Government has dropped the problematic language linked to the notion of “full autonomy” as previously defined by the UK as involving a “system capable of understanding higher level command intent”.  This previous sci-fi-esque framing centred concern on futuristic capabilities, which are a long way off and may never be realisable, thus detracting from urgent concerns about weapons on the brink of production. The new position further asserts that mechanisms of human authority, responsibility and accountability will always be present when force is used.

However, the position provides no indications of how “context appropriate human involvement” is to be assessed or understood. In this new formulation, “context appropriate human involvement” could mean almost anything and the lack of detail about where the UK draws the line amounts to the UK public being told by the military “leave it to us” to determine what is appropriate. This is emblematic of our previous concerns that this policy position, as well as the wider Defence AI Strategy, was formulated without public consultation or any attempt by ministers to have a national conversation on this issue.

What’s missing?

In the international discussion on autonomous weapons there is widespread recognition that certain factors are necessary for meaningful human control, or sufficient predictability, over autonomous systems, including:

• human control over the duration and geographic scope of an autonomous system’s operation – these are vital to making judgements about the possible effects of a system’s use.
• human understanding of the target profiles a system uses – this is vital to understanding what will trigger the system to apply force, including the possibility of false positives (targeting things that are not the intended targets) – and this is also an area where AI and machine learning raise particular concerns.

Regrettably, these factors are not mentioned in the policy.

There is also no indication of whether the UK considers it acceptable to allow machines to identify and automatically fire on human beings. The International Committee of the Red Cross (ICRC) and civil society organisations have urged states to reject autonomous systems that would apply force directly to people – but the UK policy position does not suggest that targeting humans with autonomous systems raises any particular concerns.

The absence of any detail about how the lines should be drawn makes it difficult to see how the UK can provide leadership in this space.  Indeed, the policy devotes a whole paragraph to arguing that global governance in this area will be difficult. Global governance is always challenging – but all the more so if you are approaching it without a clear vision of what needs to be achieved.

In its final section, the policy sets out a very loose set of ambitions that are notable in their refusal to suggest that new international law is needed on the issue.

The UK says that the UN Convention on Conventional Weapons (CCW) will be its primary avenue for work. But it neglects to note that a majority of states in that forum are calling for new international law or that the CCW is, by its very nature, a forum intended to create new international law, and yet the UK is one of the states that have been refusing to support a mandate for work on a new legal Protocol.  Worse still, the forum is deadlocked by its rules of procedure and is incapable of agreeing any consensus outcome in the current political climate – a political climate that needs new rules and standards over new weapon technologies, where the UK stands to benefit from the establishment of such standards.

Overall, the UK position shows at least a continued recognition that there are lines that need to be drawn to establish norms and standards around autonomy in weapons systems. But it contains none of the clarity and vision needed to suggest that the UK will have a leading role in bringing those norms and standards about.

The positive side of the indistinct UK policy is that there is little in it that should bar them from joining processes led by others. In the near future they are likely to tie themselves to US-led initiatives in the CCW (and possibly beyond), but such endeavours may be rapidly overtaken by the development of a free-standing legal treaty process within the years ahead.  We hope that the UK modifies its position and joins the growing number of states calling for strong, legally binding rules to stop lethal autonomous weapons.

We are grateful to the tirelessness of parliamentarians who have championed a more progressive UK policy on this issue and are pleased that their work, along with pressure from civil society, appears to have played a role in forcing the Government to explain their approach to this issue in much fuller terms. We look forward to working together to urge the UK to strengthen and add detail to this policy to ensure decisions over life and death are never delegated to a machine.