This post is also available in: עברית (Hebrew)
There are many inherent weaknesses that underlie existing machine learning (ML) models, opening the technology up to spoofing, corruption, and other forms of deception. Attacks on AI algorithms could result in a range of negative effects – from altering a content recommendation engine to disrupting the operation of a self-driving vehicle. As ML models become increasingly integrated into critical infrastructure and systems, these vulnerabilities become even more worrisome.
The US Defense Advanced Research Projects Agency (DARPA) has launched an initiative to address potential ML model attacks.
The Guaranteeing AI Robustness against Deception (GARD) program seeks to work ahead of safety challenges by developing a new generation of defenses against adversarial attacks on ML models.
Bruce Draper, the program manager leading GARD, said the initiative will “create a community to facilitate the open exchange of ideas, tools, and technologies that can help researchers test and evaluate their ML defenses. Our goal is to raise the bar on existing evaluation efforts, bringing more sophistication and maturation to the field.”
The scope of work involves researchers representing companies and academic institutions such as IBM, University of Chicago, etc. generating toolbox, benchmarking dataset, and training materials while making the assets available to the broader research community through a public repository.
“The goal is to help the GARD community improve their system evaluation skills by understanding how their ideas really work and how to avoid common mistakes that detract from their defense’s robustness,” Draper said. “With the Self-Study repository, researchers are provided hands-on understanding. This project is designed to give them in the field experience to help improve their evaluation skills,” according to homelandprepnews.com.
Central to the asset list is a virtual platform called Armory that enables repeatable, scalable, and robust evaluations of adversarial defenses. The Armory “testbed” provides researchers with a way to pit their defenses against known attacks and relevant scenarios. It also provides the ability to alter the scenarios and make changes, ensuring that the defenses are capable of delivering repeatable results across a range of attacks, according to DARPA.