Skip to main content
UploadLanguage (EN)Support
BusinessMobileSocial MediaMarketingTechnologyArt & PhotosCareerDesignEducationPresentations & Public SpeakingGovernment & NonprofitHealthcareInternetLawLeadership & ManagementAutomotiveEngineeringSoftwareRecruiting & HRRetailSalesServicesScienceSmall Business & EntrepreneurshipFoodEnvironmentEconomy & FinanceData & AnalyticsInvestor RelationsSportsSpiritualNews & PoliticsTravelSelf ImprovementReal EstateEntertainment & HumorHealth & MedicineDevices & HardwareLifestyle
Gregory Nelson, profile picture
Uploaded byGregory Nelson
PDF, PPTX1,041 views

Algorithmic Bias: Challenges and Opportunities for AI in Healthcare

The document discusses algorithmic bias in AI healthcare applications, emphasizing the need for responsible practices to ensure fairness and transparency in AI models. It highlights various potential biases in healthcare algorithms and stresses the importance of clinician involvement for validation. The text also calls for establishing trust through accountability, privacy protections, and rigorous evaluation of AI tools used in medical decision-making.

Data & Analytics◦
Download as PDF, PPTX
Algorithmic Bias Challenges and Opportunities for AI in Healthcare North Carolina Chapter of HIMSS Greg S. Nelson, MMCi, CPHIMS Vice President, Analytics & Strategy Vidant Health
A Data-Driven Transformation • How can Big Data Help? • Data allows us to ask new questions • Analytics allow us to identify opportunities • More data is the basis for competitive advantage • More data and agile methodologies enable us to find cost efficiencies Source: https://blog.westerndigital.com/business-agility-big-data-mindset/
• Consider a revolutionary test for skin cancer that does not work on African Americans….. • What about a model that directs poorer patients to a skilled nursing facility rather than their home as it does for wealthier patients? • Imagine an algorithm that selects nursing candidates for a multi-specialty practice—but it only selects white females.
Algorithmic bias is what we experience when a machine- learning model produces systematic errors that result in unfair outcomes.
Advertising High-income jobs are presented to men much more often than to women Source: A. Datta, M. C. Tschantz, and A. Datta. Automated experiments on ad privacy settings. Proc. Privacy Enhancing Technologies, 2015(1):92–112, 2015.
Advertising Source: L. Sweeney. Discrimination in online ad delivery. Queue, 11(3):10, 2013. Ads for arrest records are significantly more likely to show up on searches for distinctively black names
AI Use Cases Patterns or classes of AI problems... Algorithmic Medicine Clinical algorithms to drive medical practice AI Healthcare Advisors Diagnose and treat diseases Rev-Cycle/ Efficiency NLP + ML to identify revenue opportunities Diagnostic Interpretation Efficient and accurate readings of imaging studies Robotic Process Automation Automation of repetitive tasks Virtual Care Real-time remote monitoring and alerting Virtual Personal Health Assistants Augmented reality, cognitive computing, sentiment analysis, speech recognition, NLU/NLG
Discussion Fraud Healthcare Fraud Detection Care Pathways Adaptive Treatment Planning Patient Flow Patient Flow Management Augmented Intelligence Computer Assisted Diagnosis What are the (a) risks and (b) impact of getting these wrong?
Risk Based Approach to Validation • What can go wrong? • What is the impact of getting it wrong?
Clinical Decision Support Source: Stanson Health
Potential Sources of Bias People, Process, Technology Model development processes Underlying data and/or blending techniques Biases, perspective or experience of the author Application or operationalization of results All of these processes are driven by human judgments...
Integrated Health System View of Data
Holistically integrated data is the key
Incentives Data does not fully reflect the underlying diagnosis and treatment Data Collection Processes Data is shallower for segments of the population based on income (access) Affordability Personal, curated data is often biased toward those than can afford devices, apps, and tech Bias in EHR Data Collection?
Fear and Uncertainty What happens if the model is biased? How can I trust a black- box! How did you validate the model? What happens when the model is wrong?
So, what do we do? Flight or fight…
3 Questions to frame your thinking… • How do we ensure that our models are not biased? • How can we make sure that our models are explainable? • How can we engender greater trust?
Four primary tenets to guide our work… Being responsible for social mores Fairness Ensure the protection of individual privacy Privacy Transparency Understanding what decisions are made and why… Trust begins with transparency, verification, and accountability Trust
Transparency The goal is to understand the process by which an algorithmic system makes decisions, and we must ensure the model can be explained. How much can we trust the data sources we use? Do we trust the libraries, services and APIs that deliver algorithms and models? How can we demonstrate trustworthiness of the outcomes? Do we understand data transformations within pipelines? How does our solution conform to regulatory requirements and business constraints? What kind of explanation does this output require if any? Have we considered alternative data sources for a more complete picture? Are there any implications due to incomplete data? Do we know what algorithms to use for what problem? Are there any cultural differences in consuming the outputs? Have we found adversarial examples to invalidate the model? Did we engage relevant experts to validate outputs? Are we clear about the meaning of the data?
The black-box problem also poses issues for physicians, who lack insight into what the AI is actually doing. It’s not that they’re afraid of being replaced; it’s more that they’re afraid of basing decisions on information they can’t see” Source: Modern Healthcare https://www.modernhealthcare.com/indepth/artificial-intelligence-in-healthcare-makes-slow-impact/
Accuracy versus Explainability Accuracy Finding Balance Explainability
Trust Begins with transparency, verification, and accountability Transparency Verification Accountability “… clinician involvement is important no matter how smart the machines get. There is a strong need for the engagement of medical experts to validate and oversee AI algorithms in healthcare.” Dr. Wyatt Decker, CMIO The Mayo Clinic
Fairness Socially responsible—one that does not discriminate against classes of people that we would generally consider protected. Age Gender Sexual Orientation Race Ethnicity
Privacy Example of Unintended Disclosure
Privacy
Privacy
… to demonstrate how artificial intelligence tools can be used to predict unplanned hospital and skilled nursing facility admissions and adverse events… in testing innovative payment and service delivery models Source: CMS.gov (March 28, 2019) $1.65M Risk of inaction …
By 2022, the first U.S. medical malpractice case involving a medical decision made by an advanced AI algorithm will have been heard. It will not be because an algorithm produced an incorrect diagnosis. It will be due to the failure to use an algorithm that was proven to be more accurate and reliable than the human alone. Source: Gartner D&A Summit, March, 2019
C H O I C E C O N T R O L
C H O I C E A N A L Y T I C T E C H N I Q U E S L i f e c y c l e P r o c e s s e s D A T A S O U R C E S S K I L L S
C O N T R O L S E C U R I T Y & P R I V A C Y D A T A & M O D E L G O V E R N A N C E V I S I B I L I T Y D E P L O Y M E N T
Require businesses to conduct an impact assessment that covers the risk associated with algorithms’ accuracy, fairness, bias, discrimination, privacy, and security Software as a Medical Device Legal, Regulatory and Ethical Oversight The Algorithmic Accountability Act (2019) SaMD Pre-Specifications (SPS), Algorithm Change Protocol (ACP), and Good Machine Learning Practices (GMLP),
Gregory S. Nelson, June, 2019 – North Carolina Medical Journal AI governance is the process of assigning and assuring organizational accountability, decision rights, risks, policies, and investment decisions for applying artificial intelligence.
We are not looking for robots to do work for us, we are looking to make better decisions by benefiting from machine learning and AI. Manu Tandon, CIO Beth Israel Deaconess Medical Center
Extend D&A Governance to AI
Analytics Product Validation Questions to ask ourselves throughout the process Product Are we building the right product? Process Are we building the product right?
Question and Answers
@gregorysnelson linkedin.com/in/gregorysnelson greg.nelson@vidanthealth.com 919.931.4736 Contact