Download the ‘Draft elements on possible consensus recommendations in relation to the clarification, consideration and development of aspects of the normative and operational framework on emerging technologies in the area of lethal autonomous weapons systems- Revised Chair’s paper – 20 September 2021′

Transcript

[00:00:00] Uldduz Sohrabi: Hello, and welcome to Article 36 Autonomous Weapon Podcast series, where we raise a critical voice on weapons. My name is Uldduz Sohrabi and I’m your host. Joining me today for our CCW Review Conference episode are two of my colleagues, Richard Moyes, who is the managing director at Article 36, along with Elizabeth Minor, who is one of our policy advisors.

[00:00:41] If you’re new to our work and you would like to get to know us more, please visit our website at ARTICLE36.ORG, where you’ll find a wealth of information on policy issues on autonomous weapons. Before we get started with today’s discussion, which I can tell both of Richard and Elizabeth are very excited about.[00:01:00]

[00:01:00] Elizabeth Minor: Yes, definitely!

[00:01:01] Richard Moyes: Conference review, conference special. I think we’re calling it, Uldduz.

[00:01:04] Uldduz Sohrabi: Yes, it is. Review Conference special. Before we get started with our discussion, I’d love to take advantage of this opportunity to ask our listeners to sign a petition for the Stop Killer Robots campaign, a global coalition that Article 36 is coordinating. Together, we’re calling on governmental leaders around the world to launch negotiations for new international law on autonomy in weapon systems. The time is now for us to ensure human control in the use of force and to prohibit machines that target people. Now, to sign the petition, visit stop killer robots.org/now.

[00:01:47] In this particular CCW Review Conference episode, we’ll be talking about the new phase of autonomous weapons discussions, looking at the shift in conversation and bringing us up to date with the most recent developments in the CCW. Before we start exploring the agenda for the CCW review conference, let’s start with reflecting on the process so far. Elizabeth, perhaps you could take us through where we are in the timeline.

[00:02:20] Elizabeth Minor: So in the timeline of international discussions on autonomous weapons, as many of our listeners I’m sure will remember, this has been on the international agenda since 2013, at the UN in Geneva– first at the Human Rights Council and then in the Convention on certain Conventional Weapons, which we’re discussing today.

And so there’s been a format of discussion– a Group of Governmental Experts (GGE) taking place since 2015. So, states have been discussing these issues now for several years. I think there’s been a growing understanding of what’s going on in this area and what some of the responses might be that we will talk about in this podcast.

[00:02:56] Also I suppose, a reminder from our side of what we think needs to happen on this issue and our thinking on policy responses now. We’re really looking for an international legal response from governments to keep meaningful human control over the use of force and rule out systems that are targeting people.

[00:03:14] So what we wants to see is this rejection of the automation of killing in a new international law agreed by countries, which prohibits certain systems. Those that can’t be used with meaningful human control and those that target people, for example, and also that has a series of positive obligations to keep meaningful human control of other autonomous weapons systems. This is something that we’re kind of seeing increasing discussion of. And that’s certainly a point of interest that we’ll look at in this podcast, I think of how discussions have been developing.

[00:03:48] Uldduz Sohrabi: What progress have we made since the last Review Conference, as well as in 2021, specifically that we could perhaps confidently say stands out from other phases in this entire regulatory process of autonomous weapons so far? Could you perhaps take us through the highlights and what some of the discussions have achieved?

[00:04:10] Elizabeth Minor: So I think the, the current mandate that states have in these discussions under the GGE, involves them trying to generate some consensus recommendations on advancing the normative and operational framework around technology in the area of lethal autonomous weapons systems. So they’re supposed to be coming up with some recommendations on some kind of standards at the moment. Something that I think from our side, we think has stood out in the last year is a growing common understanding around the subject matter of what are we talking about here and also what an effective structure for a response to that might look like.

[00:04:46] I mentioned our recommendations in this area, but I think, a range of countries are increasingly thinking about some autonomous weapons and the problems in this area in terms of what do we really need to rule out and take off the table? And I think all countries have lines there, even though they might vary slightly.

[00:05:06] Also how do we make sure that we do keep human control or an appropriate level of human interaction in the use of increasingly autonomous weapon systems? So I think there’s, increasing discussion on basically a structure of prohibitions and positive obligations as we’d kind of frame it. And generally a kind of greater recognition in the space of autonomous weapons. You know, however we define those, we need to address this kind of two-part question of what’s unacceptable and needs to be ruled out ? And how can we ensure meaningful human control over the rest of the stuff?

[00:05:38] So I think that, generally with the CCW states have done a good job to increase understanding of what we’re talking about and how the problems might be addressed in policy terms. Obviously the next step from our perspective is that we need to kind of enact policy solutions in this area. And I think that they are becoming increasingly clear what they might be from the discussions. So that’s something that I think is a positive development.

[00:06:07] Uldduz Sohrabi: Richard, in your analysis on the last CCW meeting, you identify three categories of states in their approach to the discussion. Perhaps could you take us through this and explain what the motives of states are in their choice of behaviour. What would you say the different states are actually trying to achieve in the process?

[00:06:27] Richard Moyes: Yeah, thanks Uldduz. First of all, just building on what Elizabeth said. I think this last year, year and a half has been pretty transformative in terms of the international policy conversation on autonomous weapons. There is now, I think, a really clear sense of where the central orientation, the central conceptual approach to this issue lies. And it is in this area of a combination of prohibitions on the one hand and positive obligations on the other hand. I think that means that this year now, as we approach the review conference, definitely more than ever before– and I think really in an extremely significant way, states can probably understand each other far better. They can understand where they disagree with each other far better. And I think, it’s been a real shift in the conversation, but also in the potential to have a much more productive political conversation.

[00:07:19] In terms of thinking about where states are perhaps positioned then on this issue, I think that structure gives us the key starting point, within the landscape of states, as I would kind of read it, I mean, there’s lots of ways of reading this landscape, but I think in that paper, after the last Group of Governmental Experts meeting— it was broadly suggesting three groups. One group who were using this basic structure of prohibitions and positive obligations and were calling for a legal instrument calling for the start of negotiations working, I think in good faith to develop, an instrument that works on that structure and that builds on that structure and that embeds that structure in the normative landscape for the future.

[00:08:01] And then I think you’ve got a few other states who were working within a similar structure, but are not so committed to actually achieving a legal, outcome. And perhaps they would prefer just a political outcome. Perhaps they would prefer a political outcome that whilst it’s framed within the structure, doesn’t really contain much sort of bite to it—doesn’t really have much grip on the sort of practical decisions that states are going to have to make around the weapons that they’re developing or the choices they’re making in conflict. So I wouldn’t like to suggest that those states are operating in bad faith in a way, but part of me suspects that they would quite like to see the issue go away a little bit. And if a quick political response within this structure could provide a way of making it look as though a solution had been arrived at, they might be amenable to that. So there’s, those two groups — both working within the same basic structure of prohibitions and positive obligations regulations.

[00:08:57] That doesn’t mean within that structure, that they actually agree with each other on how to formulate the details of that. But you could perhaps recognise that there’s a structural similarities there. And then next to them, you have another group of states who are really the most disruptive. If you were looking at a classroom of school children, they’d be the children on the back sort of throwing the pencils and the pencil erasers around while the teachers try to teach. They’re refusing to buy into this kind of structure, because they know that the movement towards a shared structural understanding is ultimately a building block for political responses. I mean, there’s a reason why we like the fact that there’s this movement towards a structure because it gives us these foundations for agreement in the future. Therefore these other states have to find ways not to recognise that structure, not to agree with it, to keep talking about, you know, emerging technologies in the area of lethal autonomous weapons systems, when other states seem to be able to start talking about autonomous weapons and, you know, talking in a much more natural way.

[00:10:00] Still this other group of states are very stuck within the mandate language that we agreed in the past, and this is the most important thing as if we knew all the answers, eight or nine years ago. And that those choices of words couldn’t possibly be deviated from now, I mean, it’s all a political game, of course, but, we could see that sort of slightly disruptive element, they’re the states who are most likely to block movement to more progressive work in the future, that the states who really don’t want to see an outcome on this issue. I think the difficulty for them is it does start to look a little childish. If it’s clear that other states are starting to have kind of a grownup productive conversation within a sort of recognisable structure, and you’re still just playing games with wordings and playing games with ideas of precedent and the like. I don’t want to name names in terms of putting states into these different, groups, but…. Well, I’ll probably accidentally name names later on anyway.

[00:10:55] Elizabeth Minor: I think it would probably be clear to anyone who’s following the [00:11:00] discussions as well. And I also wonder who is the teacher in this metaphor then? Richard?

[00:11:05] Richard Moyes: Well, of course, that’s the chair. Ambassador Pecsteen, right?

[00:11:09] Uldduz Sohrabi: Richard, Elizabeth, you both mentioned increase of understanding. Can we look at some of the themes of language where we can see states are streamlining some of the issues?

[00:11:20] Richard Moyes: Well, I think to start with just the general talk around, prohibitions and positive obligations or, prohibitions and regulations. I think the repetition of that is a streamlining of the conversation. I think there’s also packaged with this and extremely important, actually a general movement towards a comfort with a basic orientation to definition where people are starting to use the terminology of autonomous weapons as a broad category of things, not things that are to be prohibited altogether, but they’re providing the space within which we’re having the conversation.

[00:12:00] And within that, a recognition that certain systems should be prohibited or should be considered unacceptable. Some states want to consider them unacceptable whilst not suggesting that that should be a prohibition. Again, this is a sort of political game playing business. I mean, we know in international law, what one does with weapons that are unacceptable, they’re the prohibited in the law, but basically recognition that within that scope of autonomous weapons broadly, some systems should be prohibited, others need to be subject to particular rules and positive obligations. I think there’s quite a lot of agreement actually on what those positive obligations might consist of. So there’s a recognition that controlling the duration and the spatial area of systems functioning is important and understanding what that system will target, understanding the target profiles that it uses or understanding the types of targets that it might function against. Those components of both system understanding, and spatial area duration constraint. I think there’s fairly broad agreement on those issues too. So I think all of that is extremely productive. There are some distinct areas of course of division perhaps, or difference within the debate as well.

[00:13:15] I mean, I think we see the prohibition on systems targeting people as being particularly important, important for moral purposes, as well as the sort of precautionary orientation to the existing legal framework. The idea of prohibiting systems that target people or that would target people is not as strongly embedded in the conversation yet as, I think, it needs to be. So that’s definitely an area where we, within civil society and other actors, need to keep doing more work.

[00:13:43] Uldduz Sohrabi: So when I hear you talk about that and also from following the issue and how the campaign has participated in the discussion, I would imagine that getting the preemptive ban on systems that target humans as being a priority. Am I right in saying that or do all the issues in the discussion take priority, parallel to one another?

[00:14:09] Richard Moyes: Everything’s a priority. I reminded myself of a joke from the mine action world, where we were trying to prioritise the clearance of minefields in a particular country. And our local partners kept telling us that every minefield was a priority, which isn’t very helpful when you’re trying to decide which one actually should be cleared first. But I think in this area, it’s a package of responses that needs to fit together as a coherent, sort of legal and sort of moral, ethical response to the challenges posed.

[00:14:38] So I think perhaps in campaigning and messaging terms the targeting of people needs to be given some greater emphasis because I think we got more ground we need to make on that side. But I think, as a response to the issue we would want to see a legal instrument that contains all of these elements, prohibitions on systems that target people, prohibitions on systems that can’t be used with meaningful human control, and then these positive obligations to ensure that people understand the systems that they’re using and that they’re constraining the duration and the spacial area of system use so that they can apply meet their existing legal obligations.

[00:15:13] Elizabeth Minor: And I think as Richard said, you really can’t have one without the others. We need all of this to work together to have a kind of effective regulatory response to this issue of increasing autonomy and weapon system. Also like you were saying, I mean, I guess the targeting people prohibition is something we want to feature more and kind of have more prominence maybe I’m in state discussions because we think it’s something that hasn’t been addressed so much, but all these things really need to fit together. And as Richard said, there’s I suppose a lot of material and discussion on meaningful human control and like aspects of that, which is quite useful for the positive obligation side.

[00:15:51] I think also we’re seeing some convergence, but also some thinking about what are systems that can’t be controlled and what the lines around that, including systems that can set their own goals or if all of their functions like during use and some kind of clear reliance of things that might be ruled out in that space. So I think even though there’s not, agreement in these areas in a straightforward way, there’s definitely progress in terms of thinking about what some of the components would be.

[00:16:19] Richard Moyes: One other thing I thought I might say maybe because I got out of bed on a very positive mood this morning for some reason.  Even though perhaps the prohibition on targeting people has a bit of catching up to do and in this discussion, it’s also, I think, one of the sharpest clearest most distinct lines in the sort of policy orientation that we and certain other actors are taking. It’s the line that sets the sort of the clearest normative, direction and sets the clearest sort of line of standard in a way.

[00:16:53] So I think that that normative clarity that’s tied up with this old notion of prohibiting systems that target people. I think that means that this issue can make up a lot of ground very quickly. Even later in the race, as it were.

[00:17:06] Uldduz Sohrabi: So the line on prohibitions, on systems that target people, we could say sets the direction of the conversation that we want to be having. Let’s get more current and look at the key issues that face the final group of governmental expert meetings of 2021, and then the review conference. So the chair handed out a paper called “draft elements and possible consensus recommendations in relation to the clarification, consideration and development of aspects of the normative and operational framework…”

[00:17:40] I’m not done… “on emerging technologies in the area of lethal autonomous weapons.”

[00:17:46] Just a note for the listeners. We have made this document available on our website at article36.org, where you can download it. Richard, perhaps you would like to talk about the content and expectations for this paper. What does it contain and what will happen to it next?

[00:18:03] Richard Moyes: Well, it’s not a snappy title for starts. I mean..

[00:18:08] Uldduz Sohrabi: Well, like you said, everything has to be going simultaneously.

[00:18:12] Richard Moyes: Exactly. And you can feel within the title of the document, the nature of the CCW in terms of its efforts to talk about things that it has to both move towards and move away from meaning at the same time. I thought this paper was very interesting. I mean, as a political approach to the subject, I felt it was initially quite a bold approach of trying to synthesise a package of materials that were starting to look a bit like what a normative framework could look like.

[00:18:42] Of course, when we say normative framework, we would like that to be a legal instrument. This paper was rather ambiguous. It was completely ambiguous as to whether it was trying to describe a legal instrument or a political instrument, frankly, it felt a bit more political instrument-ish in places which I definitely, well, definitely don’t, really appreciate. But it was quite bold to set out this package and in certain respects, it embraced the same structure that we are in favour of in terms of promoting both prohibitions and positive obligations. However, there was a number of downsides to it. The way the prohibitions were framed was extremely problematic.

[00:19:22] It drew on formulations that, I think, had been used previously by France and certain other states where it kind of initially it framed fully autonomous weapons as being unacceptable. And then it framed fully autonomous weapons systems, the systems designed to operate outside of any framework of human command and control. I mean, it’s just a fantastical kind of definition. The idea of people designing systems that are intended to operate outside of any kind of framework of subsequent human management and control. It just doesn’t really make any sense. I feel like if you turn the system on in some way, then that’s a form of human control. Right?

[00:20:04] It literally would have to refer to systems that got up halfway through the production process and walked off and you didn’t see them again. So, it’s a very, well, it’s a poor definition, but it was also, I think a deliberately poor. I mean in its origins to that formulation is deliberately poor in the sense that I think it’s an attempt to claim or to label as unacceptable something that is so preposterous that it’s never going to exist. And therefore it’s not going to have a bearing on actual practice in terms of either weapon development or the behaviours and choices of states or weapon use.

[00:20:40] So, I think I’m back to resisting my desire to suggest there is a cynical agenda at work in this, but one could feel that there is a co-opting of the term fully autonomous weapons that an attempt to apply to that label, a meaning that is totally nonsensical.

[00:20:58] And then to be able to say, and we’ve agreed that these are unacceptable and shouldn’t be adopted. That’s one problem. Another problem is it doesn’t contain anything on the prohibition of systems targeting people. Despite that being raised a number of times, there’s no sort of recognition of that.

[00:21:14] There was some good stuff on positive obligations in there. Also a lot of other content, however, in this document, which was about risk mitigation measures and reviewing new weapon systems and there was far more on how to mitigate the risks than there was any description that there were any actual risks associated with these systems.

[00:21:32] So a lot of content that really just repeated, existing practices in relation to weapons in general or existing obligations of the law. So you could probably cut it down by 50% and you wouldn’t lose any meaning in terms of content as well. And I think a lot of that content was kind of a political sock to certain states who are thinking of the UK in this area, who wants to spend the conversation talking round and round in circles about, how we already review systems and this, that, and the other. And yeah, it’s just a bit of a time-wasting mechanism, ultimately.

[00:22:06] So, the paper’s a bit imbalanced. Well, it was significantly imbalanced… There’s loads of other small issues in it, as well as some of which, relationship to human rights law and the like as well. More significantly, I guess, is in the process of the discussion at the last GGE this paper kind of as, as the way of these things got attacked from all directions really. And this is why I described it as kind of bold because I mean, one of the words in that long title that you’ve read out, Uldduz, was possible consensus recommendation as well. I mean, there’s no way this is actual possible consensus recommendations in the sense that there was no way that those things were actually going to be agreed by consensus by the group. So what starts off as a sort of paper that suggests that it might be adopted by the group as being something that we all agree on, when I say ‘we’ I’m imagining myself in a mindset of a state, where states all agree on this. While there’s just no way that states all agree on this. So this brings us onto what are the prospects for this paper?

[00:23:04] Generally in these situations, the chair is not going to be able to push through the idea that this has any kind of consensus support. So what kind of happens is that the paper gets adopted as a paper of the chair in the chair’s personal capacity. So it’s not framed as representing the views of the group, but rather as just— this individual, this person’s— best efforts to draft something that they feel is going in a constructive and helpful direction for the group as a whole. This paper could go in that direction towards a sort of chairs’ paper, but even there, I think , it’s a bit awkward. I think more likely is that it gets stripped down to a very few key remaining, probably rather bland elements and there’s an effort to sort of perhaps then fold it into something at the Review Conference that might be a more declaratory political statement for the review conference report.

[00:24:00] Anyway, this is speculating about what might happen next, which I think we’re going to do a bit more of that. So look looking forward to that in terms of the hostages to fortune in terms of making wild predictions about the future. But I don’t think there’s much chance of that paper being adopted by the group as a whole in a really meaningful substantive way.

[00:24:18] Uldduz Sohrabi: If we take a step back, what I’m hearing is that the chair handing out this paper was a step in a positive direction as you call it a bold move. Now, I’m curious, does this paper sit well with any states at this point?

[00:24:33] Richard Moyes: Yeah, probably since well with no states. I mean, which is also not uncommon in negotiations, right. That if everybody’s equally unhappy, then, you know, that’s much the same as everybody being equally happy. So I think it contains a little bit of something for everybody to dislike probably. Some of its key provisions were definitely quite similar in the linguistic formulations to papers that France and Germany had circulated not long ago. And with France chairing the review conference. Yeah. Belgium’s close to France and Germany, geographically, you know? So you know, that could be a coincidence, but there is just a sense that perhaps that’s the choice of formulations in certain key areas. It was indicative of whose papers who had been read on certain points.

[00:25:20] Uldduz Sohrabi: Let’s visit the possible outcomes of the GGE meetings. Could you perhaps take us through them?

[00:25:29] Richard Moyes: Oh, yeah. I mean, do they have to agree? This is already, I’ve got a danger of showing my ignorance, but I imagine that the chair would want the GGE to make a recommendation. They need to agree a report, which is just a report on what have they done. So they have to do that. No, that can be a basically very factual statement of they met for certain days, you know, blah, blah, blah, even, even facts like that, of course, can’t be totally taken for granted in recent times, whole meetings have happened and then it’s turned out that they haven’t happened.

[00:26:04] But it would be nice if we could agree just the basic facts of whether or not meetings actually did take place. Let’s imagine we’re in a space where we can do that. So we need to pin that down, I suppose it would be nice for the chair to be able to agree by consensus some representations of the subject matter, some points of substance and content that appeared to be new and that you could get agreement from the whole group on, or which is the interpretation of consensus in this situation.

[00:26:38] Now you could probably get consensus agreement on the fact that states still recognise that the 11 Guiding Principles still apply fully to conversations about autonomous weapons or something like that. So some reaffirmation of the guiding principles as being a thing. I’m not 100% sure beyond that how far you can really get for getting states to agree some stuff by consensus on the content, which is a little bit related to the issue of the future of the chairs paper in this context, right? That’s the most ambitious notion of what you could possibly get agreement on and that’s way away from from reality.

[00:27:19] If you were to boil that down to some key elements, maybe you can find a couple of paragraphs here and there that you could uncontroversially get agreement on, but I’m not sure what they are. And I don’t think I would personally be minded to go looking for them because I think they would be sort of asinine as to be relatively insignificant in terms of that normative significance. I’m sure some states would you know, back to that notion and we’re suggesting earlier about co-opting the term fully autonomous weapons, defining it as a way that is fantastical and implausible, and then adopting some normative assertion that that’s unacceptable.

[00:27:57] I’m sure that it’s still an aspiration of certain states for the review conference, that the review conference could take from the GGE report, a statement like that, and then agree that at the review conference as a political outcome. I don’t see that happening, frankly. I know some, well, this is where we’re onto the hostages before fortune speculating, guessing what will happen.

[00:28:15] But I find that very difficult to believe for various reasons that we could talk more about that in the next episode, maybe. So there’s that, is there some substantive content that can be agreed by consensus about it? And then there’s a recommendation regarding the mandate that should be adopted at the review conference for further work of the group of governmental experts. If indeed, that is going to happen and linked to that there’s also been these proposals. There’s been more specific proposals for an additional group of experts, like for the group of experts to appoint some super experts who are going to just provide technical guidance.

[00:28:53] I don’t see that happening. I think that’s ridiculous, frankly, a ridiculous idea. I mean, it’s just like a disenfranchising of many people within the group. And it’s completely unclear who are these experts who have all this expertise who are not already adequately represented? There’s proposals that the future work should be developing what they start to call a Montreux style document. This is a set of principles or is it a summary of how you apply existing law to private military companies, private military contractors. That’s a document that exists in the world that is for some states providing a kind of intellectual template for what this process might work towards in the future. So it’s not like a political declaration. It’s kind of a like an operational guidance document. That document was actually developed by a small group of states who were committed to working together to achieve a certain outcome. It wasn’t developed through a process of consensus of the whole they may have agreed by consensus on the document but it was only a relatively small group who had chosen to work together to produce that interpretive document. This is related to what mandate might be given to future work in this area.

[00:30:08] And, ah, it’s given that they’ve got one week to work this out and I know there’s some workshops going on in the background as well, very unclear what the answers are going to be in these areas.

[00:30:19] Uldduz Sohrabi: From article 36’s perspective, what would you say would be a success if states achieved that in the review conference?

[00:30:28] Richard Moyes: We want them to adopt a mandate to negotiate a legally binding instrument, right. We want states to agree. You know, we’ve talked about this enough, the matter is urgent. It’s important for human society actually, that we do this. We need states to recognise perhaps that they are actually the people who are going to have to do this. There aren’t some other magical experts who are going to fly in from the outside and take control of business.

[00:30:51] States need to realise their responsibilities in this space and adopt a mandate to negotiate a legally binding instrument. If alongside that they could adopt some really good language on the need for prohibitions on systems that can’t be used without meaningful human control and on systems that would target people and the need for positive obligations to ensure human control on the use of systems —that would be great too. See. I don’t think now’s the time to lower our expectations and demands. We should just keep our eyes on the prize.

[00:31:23] Uldduz Sohrabi: Absolutely. So in our next episode, we’re going to be talking about how the review conference outcomes might shape the future of development of this issue, but would you like to introduce some of the key themes for that now? Perhaps like a trailer?

[00:31:37] Elizabeth Minor: Sure. So. I mean, like we’ve been saying in this podcast, there’s increasing recognition of what policy solutions might look like in this area in terms of a regulatory structure, which as Richard was just saying, we believe should be an international legal instrument and we want to see a mandate agreed on that. That’s where we think things should go next. But I think possibly the most likely outcome from the Review Conference is not going to be that and will probably just be a further mandate to discuss stuff because there’s a range of states that Richard mentioned that kind of wants to block basically anything else.

[00:32:12] There’s also some states who have slightly different ambitions. There’s a chance of a general mandate for negotiation unspecified what that would mean being put down. We can talk about whether we think that might succeed. Probably whatever happens there’ll be a range of new crimination about who spoiled to the possibility for certain possibly unachievable consensus proposals. So we’ll have a bit of a chat next episode, probably about this blame game and for a further discussion mandate. I mean, of course that’s less than what we want, but it doesn’t mean that the work of the CCW so far has been a failure or not achieved anything, as we’ve been saying, it’s made quite a lot of developments in recent years.

[00:32:52] But it would mean that states and civil society that wants to make progress are going to have to think a bit more creatively about how to push things forward towards a legal instrument using the useful work that has been done in the CCW, but perhaps convenient conversations in other formats for this kind of critical issue of technology and society. So we can talk a bit about what form that might take as well.

[00:33:17] Richard Moyes: So Uldduz, when I listen back to this episode, after we’d recorded it, I found myself wanting to add in a quick comment from the outside. And of course, because it’s a podcast rather than a live performance, we can do that. I really just wanted to sort of suggest that I didn’t think in this first recording that we’ve fully captured the extent to which the majority of states in this discussion do you want to develop a legal instrument and are committed to moving towards negotiation. So I feel that although we’re speculating about what the outcome of the CCW will be, and we know that the CCW is often constrained by its rules of procedure. We shouldn’t lose sight of the fact that the great majority of states are moving towards a legal instrument. And that’s extremely important for us as a campaign.

[00:34:08] Uldduz Sohrabi: Richard, Elizabeth. Thank you very much for joining me. That’s all from us at Article 36 on this Review Conference Special episode. If you found this podcast useful, please share it with your network and don’t forget to subscribe to our newsletter and follow us on LinkedIn and or Twitter through our handle at article 36. Bye for now.