New Autonomous Weapon Under Debate

New Autonomous Weapon Under Debate

Photo illust. US Navy
031214-N-3236B-020 Central Command Area of Responsibly (Dec. 14, 2003) -- Marines from the 13th Marine Expeditionary Unit (13th MEU) Tank Platoon BLT 1/1 stationed at Twentynine Palms, Calif., fire the M-A1 Abrams tank during a live fire training exercise. 13th MEU is assigned to USS Peleliu (LHA 5) Expeditionary Strike Group One (ESG-1). An ESG constitutes a new naval strike force designed to equip amphibious forces with added firepower and operational capabilities. ESG-1 is currently deployed to the Commander, U.S. Fifth Fleet Area of Responsibility in support of Operations Enduring Freedom and Iraqi Freedom. U.S. Navy photo by PhotographerÕs Mate 1st Class Ted Banks. (RELEASED)

This post is also available in: heעברית (Hebrew)

The danger of autonomous weapon systems is not just that they might run rogue and kill the wrong people, but that a human overseer might blindly trust what the computer tells him — a phenomenon known as automation bias.

A new development might raise new fears of uncontrolled weapons. A US Army night vision laboratory has been working on the development of an automated turret for live-fire testing next summer.  The technology will use the lab’s Artificially Intelligent Targeting System (ATLAS), designed to detect potential targets, determine if they’re hostile and aim a 50mm cannon with superhuman speed and accuracy.

ATLAS, announced earlier this year, is an ambitious program to combine computer vision, AI, and machine learning to create ground vehicles capable of making targeting decisions and taking appropriate preparatory action at superhuman speeds.

However, ATLAS will not pull the trigger because it will not have a physical connection to the trigger mechanism, leaving the final decision to fire literally in human hands. In the future, thought, the Army might explore a less hands-on form of control, the Army’s acquisition chief, Bruce Jette told breakingdefense.com. He added that army engineers will take every safety precaution and that the final product might actually be more discriminate than a weapon relying entirely on human brainpower.

A human officer might, for example, look at surveillance data, like live imagery from a drone, and clear a platoon of robots to open fire on a whole group of targets, he said. This would appear to meet the standard of having a human “on the loop”, an approach that would require the kind of automated trigger that ATLAS currently lacks.

Even this set-up would certainly alarm activists who are already skeptical of ATLAS, arguing it could easily be modified to bypass human control.

The technology is developed upon a request from the Night Vision and Electronic Sensors Directorate, which will provide the automation, the Armaments Center the physical weapon, ammunition, and turret; both are now part of Army Futures Command.

The first phase of development will focus on making the automated system as fast as possible. It will be thoroughly tested and debugged, and safety features and human controls will be added before it’s connected to an actual weapon for a “dry fire” run without ammunition, let alone the live-fire test tentatively scheduled for June 2020.