Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Towards a definition of Lethal Autonomous Weapons


Published on

  • Be the first to comment

  • Be the first to like this

Towards a definition of Lethal Autonomous Weapons

  1. 1. 1 Towards a definition of Lethal Autonomous Weapons By Guillaume Fournier, December 2016 Unmanned combat vehicles have been used by militaries from around the world to lessen the human cost of war and to gain military advantage, for thousands of years. In ancient Greece, Thucydides recorded the use of unmanned fire ships by the defenders of Syracuse to defeat the Athenian fleet. The use of such unmanned ships continued until the 19th century. The first real advances in unmanned vehicles started with Nikola Tesla who in 1898 demonstrated a radio- controlled motorboat to a government representative. The two World Wars (the German Goliath tracked mine or the Soviet TT-26 Teletank) and the Cold War saw an increase in the development and the use of unmanned weapons. Together with the rapid evolution of artificial intelligence, robotics, and of technology in a number of areas generally, the last decade has seen significant developments in a variety of unmanned combat vehicles; Unmanned combat aerial vehicles (UCAV), unmanned ground vehicles (UGV) and Unmanned surface vehicles (USV) have been developed by the United States, China, Russia, Israel, France and the UK. UCAVs, also called drones (the MQ-9 Reaper, the MQ- 1 Predator or the X-45 for the US, the Elbit Hermes 450 for Israel, the Sperwer or the Thales Watchkeeper WK450 for France, or and the Mikoyan Skat for Russia). UGVs (the Gladiator Tactical Unmanned Ground Vehicle, XM1219 Armed Robotic Vehicle or Foster-Miller TALON SWORDS for the US Army). These vehicles have different functions ranging from reconnaissance, to combat or defense and have already been used in many of the current ongoing conflicts (Iraq, Afghanistan, Pakistan, Yemen, and Somalia). But what constitutes an Unmanned Combat Vehicle, and how such vehicles can be used, still is the subject of legal debate. Most of the current weapons systems still rely on the supervision of human operators for finding their way, targeting and shooting. They are not therefore entirely considered as autonomous. This is about to change with new ground, air and underwater weapons currently being developed by militaries around the world and that would no longer be just unmanned under human supervision, but would have some increasing degree of autonomy from human operators for their mission. They would constitute LAWS: Lethal Autonomous Weapons Systems. Some militaries already own weapons systems that possess some degree of autonomy such as various automatic defensive weapons, or “fire and forget” missiles which only require that their human operator identify the target and that then proceed on their own. The United States has the Aegis Combat System and Phalanx Weapons System at sea and the C-RAM and Patriot missile system on land. Israel has its air defense system “Iron Dome”. These latest weapons are based on land or on ships and can target and attack incoming threats on their own. Nevertheless, they are still mostly automatic: they are preprogrammed and only function within the parameters given to them. Human supervisors can stop them if necessary. They can only attack a limited type of targets and will not learn by themselves new categories of objects to attack. So far, and despite holding various meetings on the subject, international instances have been unable to agree on what should constitute the distinction between automatic and autonomous
  2. 2. 2 and on the degree of this autonomy. Automatic versus autonomous weapons are distinguished by the amount of human supervision required – or not – for their operation. Various definitions have emerged that seek to create categories by degree of independence from human supervision. Human-in-the-Loop Weapons can select targets and deliver force only with a human command; Human-on-the-Loop Weapons can select targets and deliver force under the oversight of a human operator who has the power to override their actions; and Human-out-of-the-Loop Weapons can select targets and fire without any human intervention1 . Automatic weapons are under the constant supervision and control of a human operator and are incapable of learning new information or of changing their goals or targets on their own. Today, the final decision to take, or at least to initiate or program lethal action still relies to a great extent on a human operator. And as long as such weapons are programmed to function within predetermined sets of parameters, they cannot be viewed as fully autonomous. Fully autonomous LAWS on the contrary would be able to choose a valid target, to shoot and to change or stop an attack on their own, all the while learning and evolving. This opens an entirely new set of legal, not to mention practical and reliability, issues. Since 2013, three meetings of legal experts under the auspices of the Convention on Certain Conventional Weapons (CCW) have failed to produce any internationally accepted definition of what is an autonomous weapon. The main reason for this failure has been the definition of what constitutes autonomy itself, and in what part of a weapon’s activity cycle it is applied. The ICRC which is the leading organization on International Humanitarian Law issues has proposed to define autonomous weapon systems as “any system that is capable of targeting and initiating the use of potentially lethal force without direct human supervision and direct human involvement in lethal decision-making”2 . A few States (Cuba, Ecuador, Egypt, the Holy See, and Pakistan) a coalition of NGOs such as the Campaign to Stop Killer Robots3 (52 NGOs from 24 countries) and various scientists4 have called for a preemptive ban on fully LAWS. Currently no State has talked about developing and launching fully autonomous weapons that would operate without any kind human supervision, and many states have affirmed the principle of human control over these weapons, at least at some level. But that level of human control and the form that it should take, vary greatly from one state to the other. The United Kingdom5 and France6 consider that autonomous systems should be defined as ones that operate with a total absence of human supervision or control, and no link (communication or control) with the military chain of command. France considers as a result that autonomous weapons that operate without human supervision would go against the necessity for the armed forces of maintaining command and control7 . The UK has also said that all their weapons 1 2 3 4 5 laws/statements/11April_UnitedKingdom.pdf 6$file/2016_LA WSMX_CountryPaper_France+CharacterizationofaLAWS.pdf 7 laws/statements/12April_France.pdf
  3. 3. 3 will always be under human control8 . Other States such as Switzerland have put forward a less restrictive definition of “weapons systems that are capable of carrying out tasks governed by IHL in partial or full replacement of a human in the use of force, notably in the targeting cycle”9 . The majority of States and NGOs have agreed so far on the necessity of some kind of human control over LAWS. An increasing number of States have been using the term “Meaningful Human Control”. Other States have been using terms such as “appropriate” or “effective” level of human control. The United States talks for example in the U.S. Department of Defense Directive 3000.09, Autonomy in Weapon, of “appropriate levels of human judgment”10 . And some important states like Russia and China still have to even provide any definition of autonomous weapons. The problem is that as a result of all this, there is currently no definition of what these vague terms mean and what they imply concerning the degree of human supervision, in particular on initiating, stopping or canceling any attack emanating from these weapons. Fully autonomous weapons that would operate without any kind of human control or supervision may also appear to be in contradiction with military advantage: military commanders generally want to be able to decide what, whom and where they are targeting, in order to engage specific targets that will bring the military advantage they want to gain from the attack. Their development and deployment may therefore be seen as still far away. But LAWS are going to continue evolving as artificial intelligence and technology continue to evolve, and as armed forces see them as a means of reducing troop deployment and exposure, and of providing faster response time and potential military superiority. They will have increasing degrees of autonomy, and they will bring with them, beyond the legal aspects, ethical and practical issues as well, which will have their own legal repercussions. Should man delegate to machines the ability and the will to kill other human beings? Would such weapons be able to respect the rules of International Humanitarian Law such as proportionality, distinction and humanity? Humans still remain best suited to apply judgment on such matters. What would be the consequences of any malfunctioning or hacking of such machines? Could they turn against their own troops, or against civilian populations? Who would bear responsibility? Delegating life and death decisions to machines, even ones with artificial intelligence, remains a troubling and dangerous notion. Fully autonomous weapons that would operate without any human supervision seem to be out of the question so far, but the absence of international definitions leaves significant latitude for too many countries to continue developing weapons with more and more autonomy in various stages of their operation. It is for this very reason that clearer definitions are needed now, that will in turn allow for the creation of safer international regulations on LAWS. 8 laws/statements/12April_UK.pdf 9 laws/statements/12April_Switzerland.pdf 10