Algorithmic bias in healthcare: Biased algorithms can become a matter of life and death

• Author:
• Author name

  Quantumrun Foresight
• December 2, 2021

Insight summary

Healthcare AI systems face significant challenges due to incomplete data samples, which can lead to biases in diagnosing and treating patients, particularly among underrepresented groups. The medical community's stringent data privacy practices, while necessary, further complicate the creation of diversified databases. Addressing these biases requires concerted efforts to diversify data sources, adjust algorithms for fairness, and balance the need for privacy with the benefits of comprehensive data.

Algorithmic bias in healthcare context 

Investigations into the potential bias present in healthcare AI systems have highlighted the issue of incomplete data samples as a significant concern. For instance, if an AI algorithm, which is trained to identify skin cancer, lacks sufficient reference images of the disease on darker skin tones, it could be more likely to misdiagnose individuals of color. However, this problem extends beyond AI systems. A considerable proportion of clinical trials do not adequately represent women and minority groups. This lack of representation can lead to the development of drugs and vaccines that may have more side effects for these underrepresented groups, as the trials do not fully account for their unique physiological responses.

Moreover, the broader medical community often guards healthcare data closely due to privacy concerns. This protective stance is not without reason; the public has grown wary of data sharing practices, particularly after a series of data breaches by major tech companies during the 2010s. Unfortunately, this environment of privacy makes it challenging for medical researchers to create diversified databases for their studies. It also poses a hurdle for healthcare startups aiming to build AI algorithms that are free from bias, as they often lack access to a broad range of data.

For instance, a study published by the Journal of the American Medical Association (JAMA) in 2020 revealed that most of the data used to train AI algorithms came from just three states in the US. This geographical limitation in data sources further compounds the issue of bias, as it fails to capture the diversity of patient populations across the country.

Disruptive impact 

Algorithms determine their actions based on the data they are fed. As such, the quality and diversity of the data they process directly influence their output. In healthcare, if the data samples used to train these algorithms lack diversity, it could lead to disparities in the quality of care provided to different racial and ethnic groups. For instance, if the majority of data samples are from Caucasian patients, the algorithms may not perform as effectively when diagnosing and treating people of color and other minority groups, potentially leading to suboptimal health outcomes.

Despite these concerns, healthcare networks globally are expected to increasingly incorporate AI algorithms into their operations throughout the 2020s. This integration spans across various aspects of healthcare, from administrative tasks to diagnostic procedures and treatment plans. The primary drivers behind this trend are the potential for cost reduction and the improvement of overall health outcomes on a large scale. However, this does not negate the need for addressing the potential bias against underrepresented groups.

To mitigate these biases, significant efforts are being made in the private sector to increase the diversity of genomic datasets. This effort involves collecting and incorporating more data from underrepresented groups. At the same time, healthcare professionals are collaborating with AI engineers to adjust these algorithms, aiming to provide more equitable assessments for all patient groups.

Implications of algorithmic bias in healthcare

Wider implications of algorithmic bias in healthcare may include: 

• Increased media reports of misdiagnosis and drug/vaccine side effects tied to the use of AI systems, the accumulation of which may increase public distrust in traditional healthcare and pharmaceuticals. 


• Increased public and private initiatives to fund the collection of diverse genomic databases to the development of next-generation drugs, vaccines, and AI healthcare tools.


• The development of healthcare-specific digital privacy systems that will facilitate the sharing of health information without exposing individual health records.


• Economic inefficiencies as resources may be misallocated due to biased algorithms, leading to less effective healthcare delivery and higher costs for treating misdiagnosed conditions.


• The introduction of new political debates and policy considerations as governments grapple with the need to regulate AI in healthcare to prevent bias while also fostering technological advancement.


• Technological advancements in AI as the need to address bias could drive research and development in machine learning techniques that are more capable of handling diverse data.


• New roles for AI ethics specialists and data scientists specializing in diverse data collection and analysis.


• The drive to collect more diverse data leading to increased use of electronic health records and digital devices, influencing energy consumption and electronic waste generation.

Questions to consider

• How else might AI-powered healthcare innovations be impaired by bias?


• Do you think AI systems can ever be trusted to diagnose patient health concerns?