Laura Alonso Alemany

Laura Alonso Alemany's talk highlighted the growing influence of artificial intelligence in various aspects of our lives, including justice, education, health, and freedom management. She expressed concerns about the biases and mistakes made by AI systems, emphasizing that these mistakes can have significant and costly consequences, especially for vulnerable groups.

She provided examples of AI biases, such as the Dutch tax authorities' algorithm that wrongly identified families as high fraud risks, leading to financial hardship and even suicides. She stressed the importance of addressing such mistakes, as they disproportionately affect specific groups.

Laura encouraged taking action to audit and inspect AI systems, despite common counterarguments like blaming the data or system complexity. She emphasized the need to focus on the system's behavior and behavior inspection rather than the inner workings of AI. She showcased examples of Twitter users exposing biases in image cropping algorithms.

In the context of language, Laura discussed biases in autocomplete suggestions, highlighting how certain stereotypes are perpetuated in search engine suggestions. She introduced a tool developed by Via Libre to examine and address language biases in AI systems.

Overall, Laura's talk called for vigilance in addressing AI biases and mistakes, emphasizing the importance of involving experts in discrimination and social impact in the development process to prevent harm before it occurs.

OK, so here I go. Thank you very very much for inviting us here. I am representing Via Libre, but Via Libre is full of amazing people and by working in quite important issues, mostly in Argentina, but things that are also impacting other parts of the world at some point. Via Libre was very instrumental in making electronic vote, making it evident that electronic vote should not be implemented in Argentina because it posed very difficult problems to solve. So now I am happy to be part of this artificial intelligence issues with human rights. So today I want to tell you about a little bit about the gallery of horrors that we can find in artificial intelligence. I will go quickly through this because I I understand that most of you have heard things. It's just that I want to make sure that everyone knows. And then I will talk about tools and approaches that are available for everyone, but especially to people who have an interest to work with computational tools with informatics to address these problems. So we're concerned. We are worried because artificial intelligence systems are covering now and more and more important aspects of our lives. So they are being used in justice to write sentences or to add judges to write sentences. They are used to to calculate the risk that someone commits a crime. Again, they are used in education to assign students to schools or to estimate their grades. Even that was that happened during the pandemic. They are used in health to detect cancer, for example in images or to help doctors write the reports. They are used to manage freedoms like freedom of movement. You know that facial recognition system systems that the scammers that recognize your face in the street are often used by the police to detain people and have them take them to to to have a testimony or maybe consider that they are criminal and automatic moderation systems also and also are used to control or moderate the freedom of of speech in for example social media. And recently with the advent of this CHAT GPT and DALL E, we are a little bit worried that artificial intelligence is contributing to the how much homogenization of the public space because these technologies are very western culture centered and they are producing such a huge amount of speech and images that the the production of other cultures gets invisible. So we are concerned, yeah. But what I'm going to talk about today is more precisely about mistakes that these AI systems make. Because it is worrying that the way they they work may affect our life. But it is even more worrying that they make mistakes and that these mistakes are not are not innocent. Yeah, they're very, very costly mistakes. And I want to take this example here. This was you have an URL here to if you want to read more about this case. This was the case where Dutch top tax authorities used self learning machine learning algorithm to create risk profiles to identify people who were possibly committing fraud in obtaining child care benefits. So they addressed families stolen based only because this system that had been trained in past experience said that some this particular family had the high risk of committing fraud, high risk of receipt of receiving a benefit that they didn't deserve, that that shouldn't apply to them according to the rules. And this this system made mistakes and many families were pushed to poverty because they all of a sudden, all of a sudden the tax authorities require that they pay huge amount of money. Some people even committed suicide because they couldn't affront this debt and more than 1000 children were taken into foster care. Yeah, So what people were really worried about was mistakes. No, these are if you make a mistake and take a child into foster care, or someone commits a suicide because you make a mistake, then this mistake is is very costly. So are these mistakes random or are they not? Well, apparently dual nationalities of people who were Dutch and for example, Turkish or Moroccan, and people who had low income systematically were flagged as a high risk and would be challenged to give back the childcare benefits. So this this was done without any proof that they had actually committed the the fraud. And this was encoding in the machine learning algorithm some bias that was institutionalized in the past activity of the of the tax authority. So this implied that there were many more mistakes against people with a non western appearance, people with dual nationality and people who were who who were on low income. So this is the kind of mistake that cannot be that that we need to to address. Why? Because there are some groups that are especially vulnerable and mistakes concerning those groups need to be avoided more than in in other cases. And also when we say that there there there is a bias, what we are saying is that they are more frequent. So it is not only that these groups require less mistakes because of their vulnerability, but also that there are more mistakes in these groups. The system is biased. So you would say no, but it's only an isolated case where I I am giving you here lots and lots of examples of a biased AI where where you you can you can check and and see that that there have been many other cases where this kind of behaviour where some social group has been subject to many more mistakes have happened. So, and it also happens that we are dealing with problems that are well beyond the capabilities of the systems. Like example reading mites, which is something that not not even humans could do, how can we expect a machine to do it? OK, so you will say, OK, but this happened once, twice. Anecdote is no evidence and say no, we can measure it. Yeah. And and we can see for example this is a very famous case of software that was used. It was used in the United States of America to predict, to predict the risk that someone would commit a crime again and then this was used for by judges to grant parole or not to some people. So we we could see that if we looked at the the errors in differentiating white or African American people, we could see that there were many more errors of the kind that the person was labeled high risk but didn't reoffend for African Americans than for white people. And the opposite also when also happens. So white Americans were more very often labeled more often than African American people labeled lower risk and yet they be pre offended. So what we need to do is just find this, this protected variable and then you can apply metrics that check for independence. Yeah, we're you'll find that a which is the the protected variable is independent of the result. Yeah. Whoops, sorry, this was given of the of the of the actual class given with the result. Yeah. So these are very, very, very easy metrics to apply. You can you have a full Wikipedia article on this and it can help everyone check whether something is a feeling or it is actually happening. Yeah. So I would like now to take to to to talk a little bit more about how to take action. If you want to take action to to audit or to inspect a a systems you will often be you will often find these counter arguments like OK, no, no, no, the the system is working well. It's not the system that is not working well, it's the it's the data. So in fact this happens like this not like the case of Dutch errors. You can, we could have said no, it's what happens is that Turkish and Moroccan people commit more fraud. So that's why it's it's happening. The system is predicting that. But we don't want that because we want to change. We want to protect a vulnerable people and if there is a correlation between no nationality and low income, we still want to protect people who are low income. And we do not want to discriminate against people who are systematically discriminated. So we don't care if the data says anything, because also choosing and representing data has many political decisions in the middle. So you don't care about the data, you care about the behaviour. And very often they say also it is complex, it's too difficult for you to understand. But that is not true. We will see now how and then you will, you will see the classical, OK, no, it's not. It's the machine. So you go through the machine, the judge should talk to the machine. And that is also not true. Ever behind any machine there is someone doing that. So let's take action. Let's focus on the behaviour, not necessarily on the inner working of artificial intelligence. So I am going to show you what Twitter users with no, not necessarily any any knowledge of artificial intelligence did at some point. They will at some point. Twitter tool where if you had a big photo, it automatically cropped the photo so that you could see what was more relevant. You could see a small part of the photo in the provisional section of the tweet. So this photo was cut like this. But then someone said, huh, what about if I feed the Twitter with these two photos? The result will not surprise you because of course both photos were the same way. So just by doing this, these users found that Twitter had a bias for white smiling faces. Yeah, and this not only happened, it also happened. For example, with this tool that was here you have the references. Whenever you find this blue text, it's a reference that you can click. And this tool that made a high resolution version of a low resolution picture that did things like how do you make the high resolution version of this person? Like this? Yeah. Or how do you make a high resolution version of this picture which was originally reduced from this picture like this? Or because it is not very normal or it's not very frequent in the training data set that people do not have hair or that people wear glasses? Yeah. So you can very easily see if the if a given tool is discriminating against you, but just by checking its behavior. So you can detect possible biases in errors that you systematize your exploration and then you report and you know what happened when people reported to Twitter, Twitter removed that feature and did not take action into cropping photos because it they understood that it was unethical. It was telling people that they were more invisible than other people, less relevant than other people, not women. And that is not something we want, that it's discrimination. So we can. There are some some approaches that make counter factors. This is a very, very funny explanation of how to work with counter factors in this case and this photo. An artificial intelligence system said that in this photo you could see a dress, a woman's dress. Then they removed the broom and they said this is a man's shirt. Yeah. So this kind of example counterfactuals we call them are very useful to detect either the system has some biases that you can for example use this framework, it's the what if rule and very suggesting name and you can get closer look with that. But then of course they will tell you no, no, but you know this is a very complex neural network. You cannot work with that. We cannot change that because it's so complex, because it's you know and they they will throw technical names at you. Then there are some tools to address the inner working to inspect, to audit the inner workings of artificial intelligence tools for images. We have a very very nice tools that show what part of the image an artificial intelligence is focusing on to to detect some some concepts. So for example here you can see that the blue part here is used to generate the the word zebra, then the red part here is used to generate another zebra and then this green part here is generate to it is used to generate the the dirt word. So with that it was found, for example, that woman was more strongly associated with cooking tools and men with building tools. Yeah, because you could see what the, the artificial intelligence was focusing on to to decide whether a person works a man or a or a woman. And that is not something that is the the final behaviour that the user will get to inspect. But but it it is more the the in inner decisions that are working for this artificial intelligence. And what about language? Yeah, I am a linguist. I am a computational linguist. So my work has been focused mostly in language and in India liberal. We have been working more specifically in this aspect for the last three years. We have developed a tool, I will show it to you now, but in language biases are seen, something like that. This is in Spanish and you can see tools like of course ChatGPT, but also tools that you have been using for a long time now like the autocomplete in some web search engines where you can see that when when you type something like the poor are, the autocomplete brings you things like the poor are responsible of their own poverty. The poor are poor because they chose to be. The poor are the best business in the world. So this is biased, this is a harmful stereotypes and we developed this tool that you can access by clicking here. You can access a demo here that you can test in Spanish or English as you prefer. This is a demo. Of course. There is a library behind that that you can use to implement and examine any word embedding or language model that you want, and that allows you to see things like this. For example, this is something we did when we found when. We're working with this tool with nutrition science people who found that in Spanish the word fat in feminine was very similar to the word fat ugly in feminine, but the word ugly masculine was more dissimilar to the word fat in in masculine. And this all then implies that the outer completions in the search engine were negative for fat in feminine and quite positive for fat in masculine, you know. So we don't know why this happens. But what we can see with this tool is that this happens and we can systematize it. Like we we gather words that are representing the concept of poor, but other words that are representing the concept of rich. And then we see that words like gorgeous are beautiful are more associated to words that are closer, are more likely to occur with words that are related to rich than with words that are related to poor. But then towards LED violent criminal murder, rapists are more associated. Well, now you have an example here if you want to look at it later. And then we can also do this with sentences like, let's see which preference a language model like GPT, like ChatGPT has for completing a sentence. Like the people who live in Mark are terrorists. OK, So the most, the most likely completion is the people who live in Israel are terrorists. The people who live in the United States are terrorists. The people who live in Argentina are terrorists. Colombia and Germany and South Africa. You can look at it to check many, many, many stereotypes that you can come up with. And we did this tool, I'm, I'm going to skip this, You can check this tool. You can use this tool with people who have expertise in discrimination, either because they are scientists who study that or because they have experienced it, or both, because it's a tool that does not require any technical expertise. And we think that is very important. It is very important that people with the expertise on the the impact, the social aspects can get to be in the core of the process of development of AI applications because it is these people can spot problems before they occur and not after the harm has been done. That was my talk what I had to say. I I hope I have not taken, yeah, much of.