This post is also available in:
עברית (Hebrew)
As artificial intelligence (AI) continues to advance, distinguishing between real and AI-generated content is becoming increasingly difficult. This challenge could have significant implications, such as not being able to say if certain evidence is real or not. A new study by Carnegie Mellon University and École Centrale Nantes addresses this issue by exploring the limitations of deepfake detection in environmental sounds.
The researchers developed a deep neural network detector designed to automatically classify environmental sounds (anything that is not speech or music) in recordings as either real or AI-generated. Their findings were presented on August 27, 2024, at the 32nd European Signal Processing Conference (EUSIPCO 2024) in Lyon, France, in their paper titled “Detection of Deepfake Environmental Audio.”
According to TechXplore, the detector currently recognizes seven categories of environmental sounds and demonstrated high accuracy during testing, with around 100 errors out of approximately 6,000 sounds. The errors were categorized into two types: the detector mistakenly labeled AI-generated sounds as real or real sounds as AI-generated.
To further investigate, the study involved 20 human participants who listened to the same sets of sounds identified incorrectly by the detector. Participants were asked to distinguish between real and AI-generated sounds. The results revealed that humans were only about 50% accurate in identifying the fake sounds that the detector classified as real, suggesting that these sounds might have subtle characteristics that both the detector and participants struggled to recognize. In contrast, participants correctly identified real sounds labeled as fake by the detector about 71% of the time, indicating a possible cue in these sounds that humans can detect, which the current detector fails to recognize.
Laurie Heller, Carnegie Mellon University professor of psychology, concluded that if this cue can be identified, it could improve the accuracy of AI sound detectors. The research points to the potential for developing more sophisticated AI detection tools capable of analyzing both speech and environmental sounds. Heller emphasized the importance of staying ahead of rapidly advancing AI technologies, warning that a future where AI-generated content is indistinguishable from reality could lead to significant societal challenges.
