SlideShare a Scribd company logo
1 of 13
Download to read offline
THE SHAKY FOUNDATIONS OF CLINICAL FUNCTION
MODELS: A SURVEY OF LARGE LANGUAGE MODELS AND
FOUNDATION MODELS FOR EMRS.
MEDIUM ARTICLE: HTTPS://MEDIUM.COM/@ABDULVAHED.SHAIK/THE-
SHAKY-FOUNDATIONS-OF-CLINICAL-FUNCTION-MODELS-D046AA08F737
BASED ON PAPER: HTTPS://ARXIV.ORG/PDF/2303.12961.PDF
ABDUL VAHED SHAIK
016452540
SANJOSE STATE UNIVERSITY
Two Foundation Models:
1.Clinical Language Models (CLaMs)
2.Foundation models Electronic Medical Records (FEMRs)
IN THIS ARTICLE:
*Benefits of Clinical FMs
*Training Data and Public Availability of models:
*Current Evaluation of Clinical FMs:
*Improved Evaluation Paradigms for Clinical FMs.
Introduction:
Foundation models are machine learning models which are easily
capable of performing variable tasks on large and huge datasets. FMs
have managed to get a lot of attention due to this feature of handling
large datasets. It can do text generation, video editing to protein
folding and robotics.
In case we believe that FMs can help the hospitals and patients in any
way, we need to perform some important evaluations, tests to test
these assumptions. In this review, we take a walk through Fms and
their evaluation regimes assumed clinical value.
To clarify on this topic, we reviewed no less than 80 clinical FMs built
from the EMR data. We added all the models trained on structured and
unstructured data. We are referring to this combination of structured
and unstructured EMR data or clinical data.
What is Clinical FM?
There are mainly two types of foundation models that are built
from EMR data. They are Clinical Language Models(CLaMs) and
Foundation models for EMRs(FEMRs).
Clinical Language Models(CLaMs) :
This is the first type of CLaMs, and it is a subtype of large
language models(LLMs). It has unique feature of specialization
of clinical/biomedical text.
Foundation models for Electronic Medical Records(FEMRs)
This is the second type of clinical FMs for FEMRs. These models
are always trained for all the timeline of all the events in
patient’s medical history.The Fig 1.b shows the perfect
explanation of FEMRs.A patient’s representation can be useful
as input to any type of built models like FEMR.
Figure 1. Overview of the inputs and outputs of the two main types of
clinical FMs. (a) The inputs and outputs of CLaMs. (b) The inputs and
outputs of Foundation models for FEMRs.
Benefits of Clinical FMs:
1.Clinical FMs have much better predictive performance.
2.Clinical FMs often require very less labeled data.
3.Clinical FMs enable very simpler and cheaper model deployment.
4.Clinical FMs are much more effective in handling multinomial data.
5.Clinical FMs can also handle novel interfaces for that are useful for
human-AI interaction.
Training Data and Public Availability of models:
CLaMs
Training Data: CLaMs are trained on either clinical text or biomedical
text. Almost all CLaMs trained on clinical text use a single database:
MIMIC-III, which has 2 million notes written.
Model Availability: Almost all the CLaMs are publicly accessible via
HuggingFace, etc.
FEMRs
Training data: Most of the FEMRs are trained mostly on small, publicly
available EMR datasets or a unique private health system’s EMR
database like MIMIC-III. It has less than 40,000 patients.
Model Accessibility: FEMRs lack a common process like the
HuggingFace for distributing models to the research community.
Current Evaluation of Clinical FMs:
Clinical FMs are assessed based on the tasks are relatively very easy for
evaluation. These provide very limited insight on the FMs being a
“categorically different” technology.
CLaMs
We collected almost every evaluation task which a CLaM was evaluated on its
previous original publication. These are evaluated based on standard tasks
and standard datasets.
FEMRs
We collected the real tasks on which each FEMR was evaluated in Figure 3b.
Evaluation done based on standard tasks and standard datasets. This is even
worse than that of CLaMS. FEMRs lack a huge set of “canonical” evaluations.
This makes it highly non feasible to compare the performance of various
FEMRs.
Figure 2. A depiction of CLaMs way of training, evaluation and publishing.
Figure 3. A Depiction of FEMRs and the way they are trained, evaluated and
published.
Improved Evaluation Paradigms for Clinical FMs.
1.Better Predictive Performance.
2.Less Labeled Data.
3.Simplified Model Deployment.
4.Emergent Clinical Applications.
5.So many emergent clinical applications.
6.Multimodality
7.Novel Human-AI Interfaces.
Conclusion
This review of 50 CLaMs and 34 FEMRs, shows that most of the clinical FMs are
being evaluated on the tasks which give very less information on the advantages of
FMs over the traditional ML models. Figure2 and Figure3 show that very less work
has been performed to validate if there are any other benefits of FMs.
We focused this review mostly on the benefits of clinical FMs with which we can
conclude that there are various risks involved and many disadvantages which needs
some attention. Similar to traditional ML models , FMs are also open to biases
induced by overfitting of the datasets.
Keeping all these aside, FMs are really prominent in solving huge range of
healthcare complexities.
Thank
You

More Related Content

Similar to shortstory258 slides.pptx

BioCreative2023_proceedings_instructions_authors_template.pdf
BioCreative2023_proceedings_instructions_authors_template.pdfBioCreative2023_proceedings_instructions_authors_template.pdf
BioCreative2023_proceedings_instructions_authors_template.pdfAlHayyan
 
Operation research and its application
Operation research and its applicationOperation research and its application
Operation research and its applicationpriya sinha
 
Controlling informative features for improved accuracy and faster predictions...
Controlling informative features for improved accuracy and faster predictions...Controlling informative features for improved accuracy and faster predictions...
Controlling informative features for improved accuracy and faster predictions...Damian R. Mingle, MBA
 
Interpretable Machine Learning_ Techniques for Model Explainability.
Interpretable Machine Learning_ Techniques for Model Explainability.Interpretable Machine Learning_ Techniques for Model Explainability.
Interpretable Machine Learning_ Techniques for Model Explainability.Tyrion Lannister
 
Bio-Inspired Requirements Variability Modeling with use Case
Bio-Inspired Requirements Variability Modeling with use Case Bio-Inspired Requirements Variability Modeling with use Case
Bio-Inspired Requirements Variability Modeling with use Case ijseajournal
 
IRJET - Term based Personalization of Feature Selection of Auto Filling Pa...
IRJET - 	  Term based Personalization of Feature Selection of Auto Filling Pa...IRJET - 	  Term based Personalization of Feature Selection of Auto Filling Pa...
IRJET - Term based Personalization of Feature Selection of Auto Filling Pa...IRJET Journal
 
BIO-INSPIRED REQUIREMENTS VARIABILITY MODELING WITH USE CASE
BIO-INSPIRED REQUIREMENTS VARIABILITY MODELING WITH USE CASE BIO-INSPIRED REQUIREMENTS VARIABILITY MODELING WITH USE CASE
BIO-INSPIRED REQUIREMENTS VARIABILITY MODELING WITH USE CASE mathsjournal
 
ICU Patient Deterioration Prediction : A Data-Mining Approach
ICU Patient Deterioration Prediction : A Data-Mining ApproachICU Patient Deterioration Prediction : A Data-Mining Approach
ICU Patient Deterioration Prediction : A Data-Mining Approachcsandit
 
ICU PATIENT DETERIORATION PREDICTION: A DATA-MINING APPROACH
ICU PATIENT DETERIORATION PREDICTION: A DATA-MINING APPROACHICU PATIENT DETERIORATION PREDICTION: A DATA-MINING APPROACH
ICU PATIENT DETERIORATION PREDICTION: A DATA-MINING APPROACHcscpconf
 
IMBALANCED DATASET EFFECT ON CNN-BASED CLASSIFIER PERFORMANCE FOR FACE RECOGN...
IMBALANCED DATASET EFFECT ON CNN-BASED CLASSIFIER PERFORMANCE FOR FACE RECOGN...IMBALANCED DATASET EFFECT ON CNN-BASED CLASSIFIER PERFORMANCE FOR FACE RECOGN...
IMBALANCED DATASET EFFECT ON CNN-BASED CLASSIFIER PERFORMANCE FOR FACE RECOGN...gerogepatton
 
Imbalanced Dataset Effect on CNN-Based Classifier Performance for Face Recogn...
Imbalanced Dataset Effect on CNN-Based Classifier Performance for Face Recogn...Imbalanced Dataset Effect on CNN-Based Classifier Performance for Face Recogn...
Imbalanced Dataset Effect on CNN-Based Classifier Performance for Face Recogn...gerogepatton
 
PREDICTING BANKRUPTCY USING MACHINE LEARNING ALGORITHMS
PREDICTING BANKRUPTCY USING MACHINE LEARNING ALGORITHMSPREDICTING BANKRUPTCY USING MACHINE LEARNING ALGORITHMS
PREDICTING BANKRUPTCY USING MACHINE LEARNING ALGORITHMSIJCI JOURNAL
 
A Study On Hybrid System
A Study On Hybrid SystemA Study On Hybrid System
A Study On Hybrid SystemCarmen Sanborn
 
Building_a_Readmission_Model_Using_WEKA
Building_a_Readmission_Model_Using_WEKABuilding_a_Readmission_Model_Using_WEKA
Building_a_Readmission_Model_Using_WEKASunil Kakade
 
OSS 2011 Multi-Level Modelling Presentation
OSS 2011 Multi-Level Modelling PresentationOSS 2011 Multi-Level Modelling Presentation
OSS 2011 Multi-Level Modelling PresentationTimothy Cook
 
Validation and Verification of SYSML Activity Diagrams Using HOARE Logic
Validation and Verification of SYSML Activity Diagrams Using HOARE Logic Validation and Verification of SYSML Activity Diagrams Using HOARE Logic
Validation and Verification of SYSML Activity Diagrams Using HOARE Logic ijseajournal
 
SBML FOR OPTIMIZING DECISION SUPPORT'S TOOLS
SBML FOR OPTIMIZING DECISION SUPPORT'S TOOLSSBML FOR OPTIMIZING DECISION SUPPORT'S TOOLS
SBML FOR OPTIMIZING DECISION SUPPORT'S TOOLScsandit
 
MULTI MODEL DATA MINING APPROACH FOR HEART FAILURE PREDICTION
MULTI MODEL DATA MINING APPROACH FOR HEART FAILURE PREDICTIONMULTI MODEL DATA MINING APPROACH FOR HEART FAILURE PREDICTION
MULTI MODEL DATA MINING APPROACH FOR HEART FAILURE PREDICTIONIJDKP
 

Similar to shortstory258 slides.pptx (20)

BioCreative2023_proceedings_instructions_authors_template.pdf
BioCreative2023_proceedings_instructions_authors_template.pdfBioCreative2023_proceedings_instructions_authors_template.pdf
BioCreative2023_proceedings_instructions_authors_template.pdf
 
Operation research and its application
Operation research and its applicationOperation research and its application
Operation research and its application
 
Controlling informative features for improved accuracy and faster predictions...
Controlling informative features for improved accuracy and faster predictions...Controlling informative features for improved accuracy and faster predictions...
Controlling informative features for improved accuracy and faster predictions...
 
Interpretable Machine Learning_ Techniques for Model Explainability.
Interpretable Machine Learning_ Techniques for Model Explainability.Interpretable Machine Learning_ Techniques for Model Explainability.
Interpretable Machine Learning_ Techniques for Model Explainability.
 
Bio-Inspired Requirements Variability Modeling with use Case
Bio-Inspired Requirements Variability Modeling with use Case Bio-Inspired Requirements Variability Modeling with use Case
Bio-Inspired Requirements Variability Modeling with use Case
 
IRJET - Term based Personalization of Feature Selection of Auto Filling Pa...
IRJET - 	  Term based Personalization of Feature Selection of Auto Filling Pa...IRJET - 	  Term based Personalization of Feature Selection of Auto Filling Pa...
IRJET - Term based Personalization of Feature Selection of Auto Filling Pa...
 
Sbi simulation
Sbi simulationSbi simulation
Sbi simulation
 
BIO-INSPIRED REQUIREMENTS VARIABILITY MODELING WITH USE CASE
BIO-INSPIRED REQUIREMENTS VARIABILITY MODELING WITH USE CASE BIO-INSPIRED REQUIREMENTS VARIABILITY MODELING WITH USE CASE
BIO-INSPIRED REQUIREMENTS VARIABILITY MODELING WITH USE CASE
 
ICU Patient Deterioration Prediction : A Data-Mining Approach
ICU Patient Deterioration Prediction : A Data-Mining ApproachICU Patient Deterioration Prediction : A Data-Mining Approach
ICU Patient Deterioration Prediction : A Data-Mining Approach
 
ICU PATIENT DETERIORATION PREDICTION: A DATA-MINING APPROACH
ICU PATIENT DETERIORATION PREDICTION: A DATA-MINING APPROACHICU PATIENT DETERIORATION PREDICTION: A DATA-MINING APPROACH
ICU PATIENT DETERIORATION PREDICTION: A DATA-MINING APPROACH
 
WP-2013-03-KOLMAT-GL-L
WP-2013-03-KOLMAT-GL-LWP-2013-03-KOLMAT-GL-L
WP-2013-03-KOLMAT-GL-L
 
IMBALANCED DATASET EFFECT ON CNN-BASED CLASSIFIER PERFORMANCE FOR FACE RECOGN...
IMBALANCED DATASET EFFECT ON CNN-BASED CLASSIFIER PERFORMANCE FOR FACE RECOGN...IMBALANCED DATASET EFFECT ON CNN-BASED CLASSIFIER PERFORMANCE FOR FACE RECOGN...
IMBALANCED DATASET EFFECT ON CNN-BASED CLASSIFIER PERFORMANCE FOR FACE RECOGN...
 
Imbalanced Dataset Effect on CNN-Based Classifier Performance for Face Recogn...
Imbalanced Dataset Effect on CNN-Based Classifier Performance for Face Recogn...Imbalanced Dataset Effect on CNN-Based Classifier Performance for Face Recogn...
Imbalanced Dataset Effect on CNN-Based Classifier Performance for Face Recogn...
 
PREDICTING BANKRUPTCY USING MACHINE LEARNING ALGORITHMS
PREDICTING BANKRUPTCY USING MACHINE LEARNING ALGORITHMSPREDICTING BANKRUPTCY USING MACHINE LEARNING ALGORITHMS
PREDICTING BANKRUPTCY USING MACHINE LEARNING ALGORITHMS
 
A Study On Hybrid System
A Study On Hybrid SystemA Study On Hybrid System
A Study On Hybrid System
 
Building_a_Readmission_Model_Using_WEKA
Building_a_Readmission_Model_Using_WEKABuilding_a_Readmission_Model_Using_WEKA
Building_a_Readmission_Model_Using_WEKA
 
OSS 2011 Multi-Level Modelling Presentation
OSS 2011 Multi-Level Modelling PresentationOSS 2011 Multi-Level Modelling Presentation
OSS 2011 Multi-Level Modelling Presentation
 
Validation and Verification of SYSML Activity Diagrams Using HOARE Logic
Validation and Verification of SYSML Activity Diagrams Using HOARE Logic Validation and Verification of SYSML Activity Diagrams Using HOARE Logic
Validation and Verification of SYSML Activity Diagrams Using HOARE Logic
 
SBML FOR OPTIMIZING DECISION SUPPORT'S TOOLS
SBML FOR OPTIMIZING DECISION SUPPORT'S TOOLSSBML FOR OPTIMIZING DECISION SUPPORT'S TOOLS
SBML FOR OPTIMIZING DECISION SUPPORT'S TOOLS
 
MULTI MODEL DATA MINING APPROACH FOR HEART FAILURE PREDICTION
MULTI MODEL DATA MINING APPROACH FOR HEART FAILURE PREDICTIONMULTI MODEL DATA MINING APPROACH FOR HEART FAILURE PREDICTION
MULTI MODEL DATA MINING APPROACH FOR HEART FAILURE PREDICTION
 

Recently uploaded

4. Cobus Valentine- Cybersecurity Threats and Solutions for the Public Sector
4. Cobus Valentine- Cybersecurity Threats and Solutions for the Public Sector4. Cobus Valentine- Cybersecurity Threats and Solutions for the Public Sector
4. Cobus Valentine- Cybersecurity Threats and Solutions for the Public Sectoritnewsafrica
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Mark Simos
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfAarwolf Industries LLC
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality AssuranceInflectra
 
React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...Karmanjay Verma
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Nikki Chapple
 
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...panagenda
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfpanagenda
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integrationmarketing932765
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Kaya Weers
 
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesAssure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesThousandEyes
 
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesMuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesManik S Magar
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxAna-Maria Mihalceanu
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observabilityitnewsafrica
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...BookNet Canada
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...Wes McKinney
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...itnewsafrica
 

Recently uploaded (20)

4. Cobus Valentine- Cybersecurity Threats and Solutions for the Public Sector
4. Cobus Valentine- Cybersecurity Threats and Solutions for the Public Sector4. Cobus Valentine- Cybersecurity Threats and Solutions for the Public Sector
4. Cobus Valentine- Cybersecurity Threats and Solutions for the Public Sector
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdf
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
 
React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
 
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)
 
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesAssure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
 
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesMuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance Toolbox
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
 

shortstory258 slides.pptx

  • 1. THE SHAKY FOUNDATIONS OF CLINICAL FUNCTION MODELS: A SURVEY OF LARGE LANGUAGE MODELS AND FOUNDATION MODELS FOR EMRS. MEDIUM ARTICLE: HTTPS://MEDIUM.COM/@ABDULVAHED.SHAIK/THE- SHAKY-FOUNDATIONS-OF-CLINICAL-FUNCTION-MODELS-D046AA08F737 BASED ON PAPER: HTTPS://ARXIV.ORG/PDF/2303.12961.PDF ABDUL VAHED SHAIK 016452540 SANJOSE STATE UNIVERSITY
  • 2. Two Foundation Models: 1.Clinical Language Models (CLaMs) 2.Foundation models Electronic Medical Records (FEMRs) IN THIS ARTICLE: *Benefits of Clinical FMs *Training Data and Public Availability of models: *Current Evaluation of Clinical FMs: *Improved Evaluation Paradigms for Clinical FMs.
  • 3. Introduction: Foundation models are machine learning models which are easily capable of performing variable tasks on large and huge datasets. FMs have managed to get a lot of attention due to this feature of handling large datasets. It can do text generation, video editing to protein folding and robotics. In case we believe that FMs can help the hospitals and patients in any way, we need to perform some important evaluations, tests to test these assumptions. In this review, we take a walk through Fms and their evaluation regimes assumed clinical value. To clarify on this topic, we reviewed no less than 80 clinical FMs built from the EMR data. We added all the models trained on structured and unstructured data. We are referring to this combination of structured and unstructured EMR data or clinical data.
  • 4. What is Clinical FM? There are mainly two types of foundation models that are built from EMR data. They are Clinical Language Models(CLaMs) and Foundation models for EMRs(FEMRs). Clinical Language Models(CLaMs) : This is the first type of CLaMs, and it is a subtype of large language models(LLMs). It has unique feature of specialization of clinical/biomedical text. Foundation models for Electronic Medical Records(FEMRs) This is the second type of clinical FMs for FEMRs. These models are always trained for all the timeline of all the events in patient’s medical history.The Fig 1.b shows the perfect explanation of FEMRs.A patient’s representation can be useful as input to any type of built models like FEMR.
  • 5. Figure 1. Overview of the inputs and outputs of the two main types of clinical FMs. (a) The inputs and outputs of CLaMs. (b) The inputs and outputs of Foundation models for FEMRs.
  • 6. Benefits of Clinical FMs: 1.Clinical FMs have much better predictive performance. 2.Clinical FMs often require very less labeled data. 3.Clinical FMs enable very simpler and cheaper model deployment. 4.Clinical FMs are much more effective in handling multinomial data. 5.Clinical FMs can also handle novel interfaces for that are useful for human-AI interaction.
  • 7. Training Data and Public Availability of models: CLaMs Training Data: CLaMs are trained on either clinical text or biomedical text. Almost all CLaMs trained on clinical text use a single database: MIMIC-III, which has 2 million notes written. Model Availability: Almost all the CLaMs are publicly accessible via HuggingFace, etc. FEMRs Training data: Most of the FEMRs are trained mostly on small, publicly available EMR datasets or a unique private health system’s EMR database like MIMIC-III. It has less than 40,000 patients. Model Accessibility: FEMRs lack a common process like the HuggingFace for distributing models to the research community.
  • 8. Current Evaluation of Clinical FMs: Clinical FMs are assessed based on the tasks are relatively very easy for evaluation. These provide very limited insight on the FMs being a “categorically different” technology. CLaMs We collected almost every evaluation task which a CLaM was evaluated on its previous original publication. These are evaluated based on standard tasks and standard datasets. FEMRs We collected the real tasks on which each FEMR was evaluated in Figure 3b. Evaluation done based on standard tasks and standard datasets. This is even worse than that of CLaMS. FEMRs lack a huge set of “canonical” evaluations. This makes it highly non feasible to compare the performance of various FEMRs.
  • 9. Figure 2. A depiction of CLaMs way of training, evaluation and publishing.
  • 10. Figure 3. A Depiction of FEMRs and the way they are trained, evaluated and published.
  • 11. Improved Evaluation Paradigms for Clinical FMs. 1.Better Predictive Performance. 2.Less Labeled Data. 3.Simplified Model Deployment. 4.Emergent Clinical Applications. 5.So many emergent clinical applications. 6.Multimodality 7.Novel Human-AI Interfaces.
  • 12. Conclusion This review of 50 CLaMs and 34 FEMRs, shows that most of the clinical FMs are being evaluated on the tasks which give very less information on the advantages of FMs over the traditional ML models. Figure2 and Figure3 show that very less work has been performed to validate if there are any other benefits of FMs. We focused this review mostly on the benefits of clinical FMs with which we can conclude that there are various risks involved and many disadvantages which needs some attention. Similar to traditional ML models , FMs are also open to biases induced by overfitting of the datasets. Keeping all these aside, FMs are really prominent in solving huge range of healthcare complexities.