Bias - AI-Talks.org

Cover Image: Kognitiv torzitasok, pszicholgia. By Braun Barbara. Creative Commons Attribution-Share Alike 4.0. November 6, 2019. Source: Wikimedia Common.

Anger and intolerance are the enemies of correct understanding.
– Mahatma Gandhi –

Maurício Pinheiro

Introduction

Artificial Intelligence (AI) is increasingly being integrated into various aspects of our lives, from virtual assistants to autonomous vehicles. However, there is a growing concern that AI systems may perpetuate and amplify existing biases in society. Bias in AI can be classified into two types: external and internal. External bias refers to cultural biases in the training data, while internal bias refers to the biases that are presented by the machine learning algorithms themselves. In this short paper, we will discuss these types of bias and their implications.

External Bias

External bias in AI systems can be caused by a number of factors, including cultural biases that exist in the training data used to train the system. This type of bias occurs when the training data used to teach the AI system contains implicit or explicit biases that are reflective of societal biases and stereotypes. AI systems are designed to learn by being fed large amounts of data, and if the data used to train the system is biased, the AI system may learn and perpetuate these biases.

For example, imagine an AI system that is designed to help organizations with hiring decisions. If the training data used to develop the system consists mostly of resumes from male job candidates, the system may learn to associate male names and characteristics with higher levels of qualifications, even if the resumes of female candidates contain the same or better qualifications. As a result, the AI system may make biased hiring recommendations that unfairly favor male candidates, leading to discrimination against women in the workplace. [1]

Similarly, in the case of loan approvals, if the training data used to develop an AI system contains biases against certain groups, such as people of color or those with lower income levels, the system may learn to associate these groups with higher levels of risk and make biased lending decisions that perpetuate discrimination against those groups. This can have serious consequences, leading to financial insecurity, reduced access to opportunities, and other negative outcomes for those who are unfairly discriminated against. [2]

To mitigate external bias in AI systems, it is important to use training data that is diverse and representative of the population, in order to minimize the risk of introducing biases into the system. Additionally, it is important to regularly monitor the system’s performance and identify any biases that may have been introduced during the training process, in order to make adjustments and ensure that the system remains fair and unbiased. By taking these steps, we can create AI systems that are more accurate, transparent, and inclusive, and that benefit all members of society.

Internal Bias

Internal bias, on the other hand, refers to the biases that are inherent in the machine learning algorithms themselves. Machine learning algorithms are designed to identify patterns in data, and these patterns can reflect biases in society. Bias plays a crucial role in neural networks, which are a type of machine learning algorithm used for pattern recognition and classification tasks. In a neural network, bias refers to a set of learnable parameters that allow the network to shift the activation function in order to better fit the training data. This helps the neural network to achieve a better balance between overfitting and underfitting, and to improve its generalization performance on new, unseen data. However, if the bias is not properly tuned, it can also introduce bias into the model, leading to inaccurate or unfair predictions.

Internal biases are especially concerning since they can be intentionally introduced by malicious actors who may hack into the system and manipulate the bias parameters to produce biased results. For instance, a hacker can alter the internal bias of an AI model in a way that favors a particular group or outcome. This is a serious problem, particularly in high-stakes applications such as healthcare or finance, where biased decisions can have significant consequences.

To mitigate the risks of internal bias, it is important to carefully monitor and tune the bias parameters in neural networks. This involves ensuring that the training data is representative of the true underlying distribution and that the bias parameters are not introducing unwanted bias into the model. Moreover, robust security measures must be in place to prevent unauthorized access to the system and detect any attempts to manipulate the internal bias of the model.

The image shows an example of a neuronal unit in a neural network, illustrating its main components. In the example, these are the inputs, the weights, the bias, the summation of inputs and bias, the activation function, and the output. The activation function chosen for this example is a ReLU (Rectified Linear Unit). By Brunello N. Creative Commons Attribution-Share Alike 4.0. August 12, 2021. Source: Wikimedia Commons.

Implications

The implications of biased AI are far-reaching and can have serious consequences for individuals and society as a whole. As we have seen above, biased AI can perpetuate discrimination, reinforce stereotypes, and limit opportunities for certain groups. It can also lead to the marginalization of certain communities and undermine public trust in AI systems.

Properly tuning the software internal bias in a neural network involves a number of steps. First, it is important to ensure that the training data is diverse and representative of the true underlying distribution, in order to minimize the risk of introducing external bias into the model. This can involve collecting data from a variety of sources, and taking steps to address any imbalances or biases in the data. Second, it is important to carefully select and configure the activation function and regularization parameters, which can affect the degree of internal bias in the model. Finally, it is important to regularly evaluate and monitor the performance of the model on new, unseen data, in order to detect and address any external (data) biases that may have been introduced during the training process. This can involve techniques such as cross-validation, ensemble methods, and fairness metrics, which are designed to detect and mitigate bias in machine learning models. By following these best practices, it is possible to create neural networks that are accurate, transparent, and unbiased, and that can be used to benefit society as a whole.

To mitigate bias in AI systems, it is important to take steps to ensure that the training data is diverse, representative, and unbiased. This can involve collecting data from a variety of sources, using multiple methods to collect and analyze data, and carefully selecting and preparing data to minimize the risk of introducing bias. Additionally, it is important to regularly evaluate and monitor the performance of AI systems, and to identify and address any biases that may arise during the development or deployment process.

Overall, the key to creating fair and unbiased AI systems is to be aware of the various types of bias that can impact their development and performance, and to take proactive steps to minimize or eliminate those biases. By doing so, we can create AI systems that are more accurate, transparent, and inclusive, and that benefit all members of society.

Conclusion

Bias in AI is a growing concern, and it is important to understand the different types of bias that exist. External bias is caused by cultural biases in the training data, while internal bias is inherent in the machine learning algorithms themselves. It is essential to address these biases to ensure that AI systems are fair, transparent, and unbiased. This can be achieved through diverse and representative training data, careful algorithm design, and ongoing monitoring and evaluation of AI systems. By addressing bias in AI, we can ensure that AI systems are used to benefit society as a whole, rather than perpetuating and amplifying existing inequalities.

References:

[1] “Amazon Scraps Secret AI Recruiting Tool That Showed Bias Against Women” – Reuters, October 10, 2018.

The article reports on Amazon’s decision to discontinue the use of an AI system for recruiting that showed bias against women due to the training data used to develop it. The system reportedly penalized resumes that contained terms such as “women’s,” and gave lower rankings to graduates from women’s colleges. This is an example of how bias in the training data can lead to unfair and discriminatory outcomes in AI systems.

[2] “How A Flawed Algorithm Led to $100M in Overdue Loans” – Wired, March 18, 2021.

The article reports on a case where an AI system was used to make lending decisions, but the system had a flaw that led to biased decisions against minority borrowers. The algorithm used factors such as zip code and education level, which were found to be biased against certain groups. As a result, minority borrowers were unfairly denied loans or offered higher interest rates, leading to significant financial harm. This example illustrates how biases in the training data used to develop an AI system can lead to unfair and discriminatory outcomes, particularly in lending and finance, which can have serious consequences for individuals and communities.