You are the head of the research group "Criticality of AI-based Systems" at the Weizenbaum Institute in Berlin and question current design principles for the development of intelligent systems. What is wrong with it, what should be done differently?
Diana Serbanescu: Like technology in general, intelligent systems frequently are surrounded by an aura of truth and objectivity. However, as Donna Haraway posits, "Technology is not neutral. We're inside of what we make, and it's inside of us. We’re living in a world of connections — and it matters which ones get made and unmade." It is important to remember that technologies designed and developed by people are also shapedy the power structures in places, and therefore prone to different types of biases. If ignored, these biases can lead to systemic discrimination.
The causes of biases are plural, from the lack of diversity in design and development teams to datasets reflecting existing social bias. We must ensure that the intelligent systems we use are designed following adequate criteria of fairness and accountability. As a result, the identification of cultural and racial biases as well as exploration of symbolic power structures existing within datasets are important research topics.
Another important step in making intelligent systems fair and accountable is to make sure we understand how they work, which is what explainability looks into. Inherently complex, these systems have become so opaque even for their own creators and, while the mathematics behind existing AI is well-theorised, one of today's greatest challenges arises from making particular implementations explainable to humans by providing a clear justification for any action or belief. Decoding what happens in the black box is crucial as AI has gained a vital impact on our lives, and we need to better understand its inner workings in order to gain trust. In order to avoid unintentional biases or misuse of AI systems, transparency and explainability are not only desirable, but mandatory.
You are also one of the founders of REPLICA, an "Institute for Creative Anticipation and Performing Arts", which aims to create an ethical framework for the discourse on future sociotechnological phenomena. What is your main goal, how can the arts contribute?