Convention on Certain Conventional Weapons Group of Governmental Experts on emerging technologies in the area of LAWS
U.S. Practice in the Assessment of Weapons Systems
Geneva, March 27, 2019
As delivered by Shawn Steene
Thank you Mr. Chair.
The United States would like to supplement its remarks during yesterday’s session on IHL challenges by responding to a few points raised by our colleagues and by offering a deeper exploration of U.S. practices in the assessment of weapons systems, which we hope will, to borrow from our Australian counterparts, both deepen and broaden understandings in this GGE.
Some delegations have recommended that in discussions on human judgement or human participation the GGE should consider and attempt to identify standards on these issues:
- the type and level of human supervision required for weapons
- the required minimum level of reliability
- the tolerable level of unpredictability.
Concerning these three specific issues, we believe that any standards for weapons systems would
depend on a variety of factors, including potentially the operating environment, concepts of employment and operations, and the type of system. This issue can involve very complex, technical and fact-specific questions. Additionally, in order to have meaningful discussions on these topics, the participation of technical experts would be absolutely critical. GGE participants would need to be prepared to discuss recent testing and evaluation experience challenges, ideally with specific examples. Here, we are mindful our UK’s colleague’s comments about the need to have empirical evidence to inform our deliberations, and we should not be hasty in drawing technical conclusions.
The United States believes it would be useful to have further technical discussion and further discussion about how IHL applies to the use of autonomy in weapon systems.
In the interest of furthering realistic technical discussions, we wanted to elaborate on U.S. practice in the review of weapons.
As we noted in our earlier intervention, the U.S. military’s practice of conducting legal reviews of weapons is conducted as part of broader acquisition processes and thus is only one component of a broader review process for weapons during development or acquisition and prior to the use of the weapon.
These types of weapons reviews include assessments of the operational effectiveness, operational suitability, and survivability, of weapons systems.
For example, as part of U.S. military processes, rigorous operational testing and evaluation is conducted. In the interest of furthering our discussion demonstrating what we think it rigorous practice in conducting reviews and testing of systems with autonomous systems we would like to offer a brief summary of the unclassified portion of one such assessment which was generated in 2015 as part of an initial assessment of the performance of the AN/TPQ 53. Please note this operational testing and evaluation assessment is distinct from legal reviews, which are also conducted in accordance with DoD policy.
This system is a counter-battery radar system, which is used to identify the location from which incoming rockets, artillery, and mortars were launched, which, in turn, can be used to direct counter-battery fire by an artillery system. The Q-53 is also designed to be operate in connection with the Counter-Rocket, Artillery, and Mortar System, C-RAM and can provide an additional sensor system for C-RAM to detect incoming projectiles.
The internet link for the document – the initial operational test and evaluation report, will be provided, for those delegations interested in reading in greater detail. This is not specifically a legal review of this system but we believe some remarks about this type of assessment illustrates how the United States includes IHL-compliance considerations in our weapons system development, from the outset. Rigorous operational testing, evaluation, and analysis of this particular issue includes a focus on ensuring that a system can support our forces ability to distinguish between military objectives and civilians and civilian objects. Even though this system is a radar system and not a weapon system as such, because it provides information that our forces might use to as part of a decision to conduct attacks, we believe that it is necessary to ensure that this system performs accurately and reliably, including by not falsely identifying military objectives.
The assessment upon which we are commenting had analysis and data from testing that occurred between November 2012 and March 2015. Our comments do notreflect subsequent testing, evaluation, and analysis, including refinements in light of this testing report. We will explain some of the relevant facts of the performance of this system, during our initial operational testing and evaluation. We believe that the operational performance concerns highlight DoD’s understanding that our ability to use a system in a way that enables that system to be employed consistent with the principle of distinction is of utmost importance. In a fielded system, having gone through this sort of review we would ensure proper use through our observance to the intended concept of employment and rules of engagement, tactics techniques and procedures, and other control mechanisms.
The AN/TPQ-53 provides acquisition and identification of mortar, artillery and rocket munitions. The information in this system feeds data back to other weapons systems, typically artillery, by identifying targets to be engaged. This system is human centric with autonomous functions that support a crew of five soldiers designated as Firefinder Field Artillery Radar Operators to operate the AN/TPQ-53.
The assessment reviewed the effective performance of the Q-53 for the following capabilities: probability of detection, probability of characterization, radar accuracy, false target rate, availability, & reliability. The released 2015 report has a detailed table with the performance of the system in several different modes.
Through our Initial Operational Test and Evaluation, we learned that the design and function of the Q-53 as of 2015, was operationally effective for detecting single-fired rocket, artillery, and mortar munitions. For single-fired munitions, testers assessed that the Q-53 provided accurate and consistent detection of counterfire acquisitions. The design at the time met the point of impact requirement for most mortar and artillery missions. The radar located the threat firing location point of origin within the required error rate for most conditions. However, it was assessed as having problems characterizing artillery & rockets, when operating in a certain mode. Testers assessed that it was not operationally effective for detecting volley-fired mortar munitions and a degraded accuracy for volley fired artillery.
In particular, during the operational testing and evaluation of the design of the radar at that time, concerns were raised about the rate at which “false targets” were reported. A false target occurrence was considered to be when the radar determined that a threat weapon is firing, when none is actually present. False targets can lead to units wasting time and resources by reacting to false warning and firing at locations in which enemy forces were not present.
DoD conducts comprehensive operational testing and evaluation to ensure that the machine does not have an unacceptable rate of “false targets” and more generally to ensure that the use of a particular system extends the intent of the military commander and operator. Systems that do not accurately identify targets do not support the execution of the commanders intent.
The U.S. Army recognized the issues identified by the assessment, and has a system for investigating and developing appropriate corrective action in subsequent adjustments to the proposed design. As our brief comments highlight, and the full report illustrates, the U.S. Department of Defense’s operational testing and evaluation of systems includes concerns consistent with ensuring that we use weapons that can be employed consistent with international humanitarian law. We hope that High Contracting Parties that are considering how to improve the rigor of their own weapons review processes and testing, will take the opportunity to review the detailed, publicly available report on the AN/TPQ-53 Counterfire Radar. We also hope to hear from other States to learn about their testing processes. Finally, the United States is confident that as High Contracting Parties consider the purposes and objectives of the CCW and how we can strengthen compliance with international humanitarian law, Parties will continue to improve the development and fielding of their systems in a manner which will minimize the risk to civilians and enable their Armed Forces to use effective and suitable weapons systems.
The 2015 assessment can be found on this website: