ARTIFICIAL INTELLIGENCE AND BIAS: LIMITS AND OPPORTUNITIES

Datasets cleaning and algorithms role to avoid racial, gender and orientation discrimination in algorithm learning process

Abstract: The article aims to prompt reflection on the crucial role of data in the field of artificial intelligence (AI), emphasizing the necessity of using bias-free datasets to preserve the performance and results produced by algorithms. Through the exploration of two main categories of biases – those introduced during the algorithm’s design and those intrinsic to the data – concrete examples of racial and gender discriminations in sectors such as healthcare, justice, and employment are presented. A thoughtful reflection, beyond the often alarmist headlines reported by the media, highlights that these distortions are not inherent to AI but rather a reflection of pre-existing biases in society, which may emerge and sometimes be amplified by algorithms. Therefore, adopting critical and constructive attitudes towards technology, without being technophobic, might be an investment in humanity and society, triggering a virtuous cycle useful for integrating technological progress and social development.

Keywords: #BiasInData #AIethics #AlgorithmicDiscrimination #TechnologyAndInclusion #SocialProgress #AIChallenges #RacialAndGenderBiasInAI #ResponsibleAI #InclusiveData #TechnologicalEthics #FairInnovation #AlgorithmTransparency #fiorenzasuccu #ethicasocietas #ethicasocietasreview #socialsciences #humasciences

italian version

Fiorenza Succu: aerospace engineer, master in business administration, worked for Eni, Bulgari, Mediaset Group, contributed to the essay “The magic of resilience”.

Algorethics: a term that has been developed since 2018 to denote the need for a study dedicated to assessing the ethical implications of technologies, particularly artificial intelligence.

GENERATIVE AI AND THE ROLE OF CHATGPT

All algorithms within the artificial intelligence (AI) ecosystem require the same fuel to function: data. Their strategic role, crucial enough to influence the performance and limitations of our algorithms, highlights the importance of having access to “clean” datasets, free from vices or biases. These biases manifest in various forms, which, simplifying, can be categorized into two main types: the first includes those introduced during the algorithm’s design operations, while the second encompasses those inherently present in the data.

During the design and set-up of the algorithm, the choice of model and/or optimization parameters, for example, may not be suitable for handling certain types of data/problems, consequently leading to suboptimal performances. Furthermore, the composition of the design team, if not harmonious and balanced, could result in an inadequate representation of different perspectives, indirectly compromising the decision-making process.

Given the inherent distortions in the data, there are numerous types that would require a comprehensive discussion, but I will mention only a few. If outdated historical data were used, the produced predictions might be inconsistent with the current reality, leading to temporal biases. Additionally, using non-representative data samples of the reference population could cause the model to struggle with performing suitable generalizations in new contexts/situations, resulting in examples of data underrepresentation and a subsequent selection bias.

Finally, in the case of supervised learning, a technique used for training algorithms, so-called labeling biases might emerge. These biases are tied to the methods of assigning labels to training data; if the assigned labels are influenced by biases of any nature (cultural, social, etc.), the model can learn, replicate, and even amplify such biases. This could have important consequences in sectors such as healthcare, justice, finance, education, and the workforce.

Several cases in these sectors have gained notoriety in the media, initially portraying technology and algorithms as perpetrators of discriminatory actions of various kinds.

CASE STUDIES: RACIAL AND GENDER BIAS

One of the most emblematic and discussed cases involved an algorithm created by OPTUM, widely used in the U.S. healthcare system to identify patients with chronic high-risk diseases eligible for extra care. Research [1] accused the model of clear racial discrimination, with the bias originating from the choice of the parameter classifying patients’ needs not based on the severity of their illness but on the cost of their past medical treatments.

Further analysis of U.S. healthcare data revealed that, generally, black patients or ethnic minorities had received inferior care over the years compared to whites. Consequently, using per capita spending as a parameter led to the wrongful discrimination of black patients by the technology. Upon re-examining all data and using illness severity as a parameter, the percentage of black patients eligible for specialized extra care programs increased from 17.7% to 46.5%. Therefore, the algorithm did not deliberately discriminate against patients of colour; rather, racial discrimination was a byproduct of an inadequate and unfair healthcare system.

Another example of racial bias involves the COMPAS algorithm (Correctional Offender Management Profiling for Alternative Sanctions), used in some U.S. judicial systems to predict, among other things, the likelihood of an offender’s recidivism. Studies revealed that, in the case of black offenders, the number of false positives for recidivism compared to Caucasian defendants was about double (with a percentage of about 45% for the former compared to 23% for the latter) [2]. Once again, as in the previous case, algorithmic biases reflect societal biases, and racial prejudice appears intrinsic in a judicial system that does not treat minorities fairly. Unfortunately, numerous high-profile incidents in the news support this reflection.

Moving on to the employment sector, specifically to corporate recruiting procedures, another type of bias emerged, this time related to gender. In 2018, Amazon reportedly halted an AI-based personnel selection program after discovering that the algorithm was discriminatory against women. Amazon’s model was designed to select candidates based on a training conducted on the resumes submitted to the company over the last ten years, among which there was a significantly higher percentage of male applicants.

Once again, the system favoured male over female candidates, but since the number of women employed in tech has historically always been lower than that of men, the algorithm merely reflected reality. The path toward equal gender treatment remains an obstacle race where women, as well as genders different from male from birth, face greater challenges when looking at the big picture.

AN AWARENESS CAMPAIGN

There could be many more examples, but I would like to conclude this article by referring to a campaign by Wired Italia released about a year ago, signed by TBWA\Italia, a part of the agency of the same name operating globally. The campaign aimed to raise awareness among readers about the content generated by artificial intelligence and its biases [4].

The focus of the campaign was a test conducted on Midjourney, a generative AI software capable of creating images based on user prompts. Instructions were given in English because, unlike Italian, this language does not assign gender to most nouns related to professions and social roles, allowing the testing of the algorithm’s gender biases.

For instance, the word “manager” could refer to either a woman or a man, but Midjourney only displayed images of white men; the prompt “lovers” was associated with exclusively heterosexual couples, and the same occurred with “parents. The prompt “scientist” displayed only white cisgender men, excluding all others, and the same fate befell the prompt “leader”, with portraits representing exclusively white men.

CONCLUSIONS AND FUTURE PROSPECTS

All the examples mentioned above share a common framework: the results of AI reflect the characteristics, modes of interaction, and thinking patterns of humans and, consequently, the society to which we all belong, with its preconceptions and distortions. Machine learning models, a subset of AI known as Machine Learning, lack an intrinsic understanding of data quality and the proposed output solutions.

The interpretation or attribution of meaning always falls upon the humans who design, train, and utilize these models. Therefore, investing in technological advancement should go hand in hand with progress in the human and social spheres. Using technology critically and constructively could offer us the opportunity to discover and address our own limitations. Reprogramming and correcting technology could turn it into an ally for our growth, triggering a virtuous cycle that could lead to continuous improvement. So, changing perspective, looking at machines and algorithms with new lenses, without succumbing to technophobic attitudes, which often stem from an inability to truly understand the technology itself, could provide us with the opportunity to grow both as individuals and as a community. Missing out on such an opportunity would be a real shame!

[1] https://www.science.org/doi/10.1126/science.aax2342

[2] https://www.nytimes.com/2017/10/26/opinion/algorithm-compas-sentencing-bias.html

[3] https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G/

[4] https://www.wired.it/intelligenza-artificiale-stupidita-umana-twba-italia-wired-campagna/