The central problem is not abstract bias but a pipeline failure that converts incomplete data and opaque model design into clinical and coverage decisions with real consequences. In healthcare, the article shows bias arising from both technical flaws and human choices, which matters because it shifts responsibility from inevitability to governance. That distinction is crucial for practitioners: if the inputs, targets, and validation logic are wrong, the system can systematically misread patient need and reinforce unequal treatment rather than merely reflect it.
The clearest mechanism is skewed training data paired with unexamined assumptions about race, demographics, or eligibility. The VBAC example is especially revealing: an algorithm predicted poorer outcomes for minority patients, which then pushed more cesarean procedures than evidence supported, before the model was revised to remove race and ethnicity. Similar risks appear in heart-failure, cardiac-surgery, and drug-development settings when limited populations are treated as representative. For practitioners, the operational lesson is that diversity in datasets and stakeholder review directly affects model validity.
The strongest corrective measures are also the least marketable: slower rollout, continuous monitoring, audits, and disclosure of how a model was built and trained. Those controls add time and reduce black-box convenience, but they are the only defensible way to detect drift and performance gaps across groups. The limits are clear too: proprietary systems may resist transparency, and fairness metrics cannot substitute for sound design. The practical significance is that governance, not branding, determines whether these systems widen inequities or support better care.
Artificial intelligence has been used to spot bias in healthcare, such as a lack of darker skin tones in dermatologic educational materials, but AI has been the cause of bias itself in some cases.ย ย
When AI bias occurs in healthcare, the causes are a mix of technical errors as well as real human decisions, according to Dr. Marshall Chin, professor of healthcare ethics in the Department of Medicine at the University of Chicago. Chin co-chaired a recent governmentโฏpanelโฏon AI bias.ย
โThis is something that we have control over,โ Chin tells InformationWeek. โIt’s not just a technical thing that is inevitable.โย
In 2023, a class action lawsuit accused UnitedHealth of illegally using an AI algorithm to turn away seriously ill elderly patients from care under Medicare Advantage. The lawsuit blamed naviHealthโs nH Predict AI model for inaccuracy. UnitedHealth told StatNews last year that the naviHealth care-support tool is not used to make determinations. โThe lawsuit has no merit, and we will defend ourselves vigorously,โ the company stated.ย
Other cases of potential AI bias involved algorithms studying cases of heart failure, cardiac surgery, and vaginal birth after cesarean delivery (VBAC), in which an AI algorithm led Black patients to get more cesarean procedures than were necessary, according to Chin. The algorithm erroneously predicted that minorities were less likely to have success with a vaginal birth after a C-section compared with non-Hispanic white women, according to the US Department of Health and Human Services Office of Minority Health.ย ย
โIt inappropriately had more of the racial minority patients having severe cesarean sections as opposed to having the vaginal birth,โ Chin explains. โIt basically led to an erroneous clinical decision that wasn’t supported by the actual evidence base.โย
Related:Why AIโs Slower Pace in Healthcare Is as It Should Be
After years of research, the VBAC algorithm was changed to no longer consider race or ethnicity when predicting which patients could suffer complications from a VBAC procedure, HHS reported.ย
โWhen a dataset used to train an AI system lacks diversity, that can result in misdiagnoses, disparities in healthcare, and unequal insurance decisions on premiums or coverage," explains Tom Hittinger, healthcare applied AI leader at Deloitte Consulting.ย
โIf a dataset used to train an AI system lacks diversity, the AI may develop biased algorithms that perform well for certain demographic groups while failing others,โ Hittinger says in an email interview. โThis can exacerbate existing health inequities, leading to poor health outcomes for underrepresented groups.โย
Related:Metaverse: The Next Frontier in Healthcare?
AI Bias in Drug Developmentย
Although AI tools can cause bias, they also bring more diversity to drug development. Companies such as BioPhy study patterns in patient populations to see how people respond to different types of drugs.ย ย
The challenge is to choose a patient population that is broad enough to offer a level of diversity but also bring drug efficacy. However, designing an AI algorithm to predict patient populations may result in only a subset of the population, explains Dave Latshaw II, PhD, cofounder ofโฏBioPhy.ย ย
โIf you feed an algorithm that’s designed to predict optimal patient populations with only a subset of the population, then it’s going to give you an output that only recommends a subset of the population,โ Latshaw tells InformationWeek. โYou end up with bias in those predictions if you act on them when it comes to structuring your clinical trials and finding the right patients to participate.โย
Therefore, health IT leaders must diversify their training sets when teaching an AI platform to avoid blindness in the results, he adds.ย ย ย
โThe dream scenario for somebody who’s developing a drug is that they’re able to test their drug in nearly any person of any background from any location with any genetic makeup that has a particular disease, and it will work just the same in everyone,โ Latshaw says. โThat’s the ideal state of the world.โย
Related:Connected Healthcare Takes Huge Leap Forward
How to Avoid AI Bias in Healthcareย
IT leaders should involve a diverse group of stakeholders when implementing algorithms. That involves tech leaders, clinicians, patients, and the public, Chin says.ย ย
When validating AI models, IT leaders should include ethicists and data scientists along with clinicians, patients, and associates, which are nonclinical employees, staff members, and contractual workers at a healthcare organization, Hittinger says. ย
When multiple teams roll out new models, that can increase the time required for experimentation and lead to a gradual rollout along with continuous monitoring, according to Hittinger.ย
โThat process can take many months,โ he says.ย ย
Many organizations are using proprietary algorithms, which lack an incentive to be transparent, according to Chin. He suggests that AI algorithms should have labels like a cereal box explaining how algorithms were developed, how patient demographic characteristics were distributed, and the analytical techniques used.ย ย
โThat would give people some sense of what this algorithm is, so this is not a total black box,โ Chin says.ย ย
In addition, organizations should audit and monitor AI systems for bias and performance disparities, Hittinger advises.ย ย
โOrganizations must proactively search for biases within their algorithms and datasets, undertake the necessary corrections, and set up mechanisms to prevent new biases from arising unexpectedly,โ Hittinger says. โUpon detecting bias, it must be analyzed and then rectified through well-defined procedures aimed at addressing the issue and restoring public confidence.โย
Organizations such as Original Postages/deloitte-analytics/solutions/ethics-of-ai-framework.html" target="_blank" rel="noopener">Deloitte offer frameworks to provide guidance on how to maintain ethical use of AI.ย ย
โOne core tenet is creating fair, unbiased models and this means that AI needs to be developed and trained to adhere to equitable, uniform procedures and render impartial decisions,โ Hittinger says.ย ย
In addition, healthcare organizations can adopt automated monitoring tools to spot and fix model drift, according to Hittinger. He also suggests that healthcare organizations form partnerships with academic institutions and AI ethics firms.ย ย
Dr. Yair Lewis, chief medical officer at AI-powered primary-care platform Navina, recommends that organizations establish a fairness score metric for algorithms to ensure that patients are treated equally.ย ย
โThe concept is to analyze the algorithmโs performance across different demographics to identify any disparities,โ Lewis says in an email interview. โBy quantifying bias in this manner, organizations can set benchmarks for fairness and monitor improvements over time.โย
Enjoyed this article? Sign up for our newsletter to receive regular insights and stay connected.