AI and Discrimination – A Study

This post is also available in: עברית (Hebrew)

A recent study by the DHBW Stuttgart looked into AI’s ability to recognize discriminatory content in images and advertisements, showing both impressive progress and existing limitations.

As part of this study, AI was confronted with various images and advertisements and asked to evaluate them- including 60 advertisements that were recently criticized by the German Advertising Council. The results show that AI can identify discriminations in advertisements with impressive accuracy and has generally assessed the advertisements criticized by the German Advertising Council as potentially discriminatory.

According to Techxplore, the rapid progress of AI raises the question of where its current limits lie. Student Helen Beckers spoke about the approach of the study: “We wanted to know to what extent AI recognizes discriminatory behavior when it is only presented with an advertisement and asked to evaluate it.”

This matter is especially relevant in light of the massive increase in discrimination by algorithms that can disadvantage people based on gender, religion, ideology, racism, or origin. Furthermore, this aspect is becoming more important because such discrimination by algorithms is becoming an increasingly serious problem that affects many areas such as application processes, credit allocation, medicine, and the calculation of an offender’s recidivism probability.

A recent revealing insight is that ChatGPT can detect sexualization and stereotypical thinking- while a modified advertisement with the gender roles swapped showed that AI can also identify discrimination in reversed situations.

However, the study found that AI reaches its limits in identifying other forms of discrimination, like objectification, disrespect, and abuse of power. The study seems to raise important questions about how AI technologies can be used in the future to combat discrimination in various areas and promote equality, and underscore the need to further develop AI systems to recognize discrimination more effectively and prevent it.