This post is also available in:
עברית (Hebrew)
AI technology is spreading quickly throughout many different industries, and its integration depends on users’ trust and safety concerns. This matter becomes complicated when the algorithms powering AI-based tools are vulnerable to cyberattacks that could have detrimental results.
Dr. David P. Woodruff from Carnegie Mellon University and Dr. Samson Zhou from Texas A&M University are working to strengthen the algorithms used by big data AI models against attacks.
Their research focuses on a type of big data model called “a streaming model” with which information must be immediately gathered from the data or it will be lost because all the data cannot be stored (such as apps that provide real-time information, like a public transportation app).
According to Techxplore, one main challenge that comes up when trying to create a secure algorithm is randomness – if an algorithm is a set of instructions for AI, then randomness is included in these instructions to save space. However, this way the engineers of an algorithm don’t have a complete picture of the algorithm’s inner workings, leaving the algorithm open to attack. Woodruff explains: “Any algorithm that uses randomness can be attacked because the attacker kind of learns your randomness through its interaction with you… And if [the attacker] knows something about your randomness, it can find things to feed your algorithm and force it to fail.”
Two main possible types of attacks are a black box attack (in which the attackers are familiar with how an algorithm responds to queries and so they base future queries on the algorithm’s previous output) and a white box attack (in which attackers know the entire state of the algorithm, its inner workings and how it responds). The researchers claim they want to defend against both types.
The researchers are creating an algorithm secure against attacks and developing new connections between mathematics and theoretical computer science, as well as looking into data encryption. They conclude that they hope to understand how to strengthen algorithms against attack while maintaining efficiency and identify principles underlying vulnerabilities in algorithms.
