SlideShare a Scribd company logo
THE SHAKY FOUNDATIONS OF CLINICAL FUNCTION
MODELS: A SURVEY OF LARGE LANGUAGE MODELS AND
FOUNDATION MODELS FOR EMRS.
MEDIUM ARTICLE: HTTPS://MEDIUM.COM/@ABDULVAHED.SHAIK/THE-
SHAKY-FOUNDATIONS-OF-CLINICAL-FUNCTION-MODELS-D046AA08F737
BASED ON PAPER: HTTPS://ARXIV.ORG/PDF/2303.12961.PDF
ABDUL VAHED SHAIK
016452540
SANJOSE STATE UNIVERSITY
Two Foundation Models:
1.Clinical Language Models (CLaMs)
2.Foundation models Electronic Medical Records (FEMRs)
IN THIS ARTICLE:
*Benefits of Clinical FMs
*Training Data and Public Availability of models:
*Current Evaluation of Clinical FMs:
*Improved Evaluation Paradigms for Clinical FMs.
Introduction:
Foundation models are machine learning models which are easily
capable of performing variable tasks on large and huge datasets. FMs
have managed to get a lot of attention due to this feature of handling
large datasets. It can do text generation, video editing to protein
folding and robotics.
In case we believe that FMs can help the hospitals and patients in any
way, we need to perform some important evaluations, tests to test
these assumptions. In this review, we take a walk through Fms and
their evaluation regimes assumed clinical value.
To clarify on this topic, we reviewed no less than 80 clinical FMs built
from the EMR data. We added all the models trained on structured and
unstructured data. We are referring to this combination of structured
and unstructured EMR data or clinical data.
What is Clinical FM?
There are mainly two types of foundation models that are built
from EMR data. They are Clinical Language Models(CLaMs) and
Foundation models for EMRs(FEMRs).
Clinical Language Models(CLaMs) :
This is the first type of CLaMs, and it is a subtype of large
language models(LLMs). It has unique feature of specialization
of clinical/biomedical text.
Foundation models for Electronic Medical Records(FEMRs)
This is the second type of clinical FMs for FEMRs. These models
are always trained for all the timeline of all the events in
patient’s medical history.The Fig 1.b shows the perfect
explanation of FEMRs.A patient’s representation can be useful
as input to any type of built models like FEMR.
Figure 1. Overview of the inputs and outputs of the two main types of
clinical FMs. (a) The inputs and outputs of CLaMs. (b) The inputs and
outputs of Foundation models for FEMRs.
Benefits of Clinical FMs:
1.Clinical FMs have much better predictive performance.
2.Clinical FMs often require very less labeled data.
3.Clinical FMs enable very simpler and cheaper model deployment.
4.Clinical FMs are much more effective in handling multinomial data.
5.Clinical FMs can also handle novel interfaces for that are useful for
human-AI interaction.
Training Data and Public Availability of models:
CLaMs
Training Data: CLaMs are trained on either clinical text or biomedical
text. Almost all CLaMs trained on clinical text use a single database:
MIMIC-III, which has 2 million notes written.
Model Availability: Almost all the CLaMs are publicly accessible via
HuggingFace, etc.
FEMRs
Training data: Most of the FEMRs are trained mostly on small, publicly
available EMR datasets or a unique private health system’s EMR
database like MIMIC-III. It has less than 40,000 patients.
Model Accessibility: FEMRs lack a common process like the
HuggingFace for distributing models to the research community.
Current Evaluation of Clinical FMs:
Clinical FMs are assessed based on the tasks are relatively very easy for
evaluation. These provide very limited insight on the FMs being a
“categorically different” technology.
CLaMs
We collected almost every evaluation task which a CLaM was evaluated on its
previous original publication. These are evaluated based on standard tasks
and standard datasets.
FEMRs
We collected the real tasks on which each FEMR was evaluated in Figure 3b.
Evaluation done based on standard tasks and standard datasets. This is even
worse than that of CLaMS. FEMRs lack a huge set of “canonical” evaluations.
This makes it highly non feasible to compare the performance of various
FEMRs.
Figure 2. A depiction of CLaMs way of training, evaluation and publishing.
Figure 3. A Depiction of FEMRs and the way they are trained, evaluated and
published.
Improved Evaluation Paradigms for Clinical FMs.
1.Better Predictive Performance.
2.Less Labeled Data.
3.Simplified Model Deployment.
4.Emergent Clinical Applications.
5.So many emergent clinical applications.
6.Multimodality
7.Novel Human-AI Interfaces.
Conclusion
This review of 50 CLaMs and 34 FEMRs, shows that most of the clinical FMs are
being evaluated on the tasks which give very less information on the advantages of
FMs over the traditional ML models. Figure2 and Figure3 show that very less work
has been performed to validate if there are any other benefits of FMs.
We focused this review mostly on the benefits of clinical FMs with which we can
conclude that there are various risks involved and many disadvantages which needs
some attention. Similar to traditional ML models , FMs are also open to biases
induced by overfitting of the datasets.
Keeping all these aside, FMs are really prominent in solving huge range of
healthcare complexities.
Thank
You

More Related Content

Similar to shortstory258 slides.pptx

Simulated Patient Care Pathways
Simulated Patient Care PathwaysSimulated Patient Care Pathways
Simulated Patient Care Pathways
Kevin Russell
 
BioCreative2023_proceedings_instructions_authors_template.pdf
BioCreative2023_proceedings_instructions_authors_template.pdfBioCreative2023_proceedings_instructions_authors_template.pdf
BioCreative2023_proceedings_instructions_authors_template.pdf
AlHayyan
 
Operation research and its application
Operation research and its applicationOperation research and its application
Operation research and its application
priya sinha
 
Controlling informative features for improved accuracy and faster predictions...
Controlling informative features for improved accuracy and faster predictions...Controlling informative features for improved accuracy and faster predictions...
Controlling informative features for improved accuracy and faster predictions...
Damian R. Mingle, MBA
 
Interpretable Machine Learning_ Techniques for Model Explainability.
Interpretable Machine Learning_ Techniques for Model Explainability.Interpretable Machine Learning_ Techniques for Model Explainability.
Interpretable Machine Learning_ Techniques for Model Explainability.
Tyrion Lannister
 
Bio-Inspired Requirements Variability Modeling with use Case
Bio-Inspired Requirements Variability Modeling with use Case Bio-Inspired Requirements Variability Modeling with use Case
Bio-Inspired Requirements Variability Modeling with use Case
ijseajournal
 
IRJET - Term based Personalization of Feature Selection of Auto Filling Pa...
IRJET - 	  Term based Personalization of Feature Selection of Auto Filling Pa...IRJET - 	  Term based Personalization of Feature Selection of Auto Filling Pa...
IRJET - Term based Personalization of Feature Selection of Auto Filling Pa...
IRJET Journal
 
Sbi simulation
Sbi simulationSbi simulation
Sbi simulation
Egart'z Sarawak
 
BIO-INSPIRED REQUIREMENTS VARIABILITY MODELING WITH USE CASE
BIO-INSPIRED REQUIREMENTS VARIABILITY MODELING WITH USE CASE BIO-INSPIRED REQUIREMENTS VARIABILITY MODELING WITH USE CASE
BIO-INSPIRED REQUIREMENTS VARIABILITY MODELING WITH USE CASE
mathsjournal
 
ICU Patient Deterioration Prediction : A Data-Mining Approach
ICU Patient Deterioration Prediction : A Data-Mining ApproachICU Patient Deterioration Prediction : A Data-Mining Approach
ICU Patient Deterioration Prediction : A Data-Mining Approach
csandit
 
ICU PATIENT DETERIORATION PREDICTION: A DATA-MINING APPROACH
ICU PATIENT DETERIORATION PREDICTION: A DATA-MINING APPROACHICU PATIENT DETERIORATION PREDICTION: A DATA-MINING APPROACH
ICU PATIENT DETERIORATION PREDICTION: A DATA-MINING APPROACH
cscpconf
 
WP-2013-03-KOLMAT-GL-L
WP-2013-03-KOLMAT-GL-LWP-2013-03-KOLMAT-GL-L
WP-2013-03-KOLMAT-GL-L
Nicholas C. Pennycuick
 
IMBALANCED DATASET EFFECT ON CNN-BASED CLASSIFIER PERFORMANCE FOR FACE RECOGN...
IMBALANCED DATASET EFFECT ON CNN-BASED CLASSIFIER PERFORMANCE FOR FACE RECOGN...IMBALANCED DATASET EFFECT ON CNN-BASED CLASSIFIER PERFORMANCE FOR FACE RECOGN...
IMBALANCED DATASET EFFECT ON CNN-BASED CLASSIFIER PERFORMANCE FOR FACE RECOGN...
gerogepatton
 
Imbalanced Dataset Effect on CNN-Based Classifier Performance for Face Recogn...
Imbalanced Dataset Effect on CNN-Based Classifier Performance for Face Recogn...Imbalanced Dataset Effect on CNN-Based Classifier Performance for Face Recogn...
Imbalanced Dataset Effect on CNN-Based Classifier Performance for Face Recogn...
gerogepatton
 
PREDICTING BANKRUPTCY USING MACHINE LEARNING ALGORITHMS
PREDICTING BANKRUPTCY USING MACHINE LEARNING ALGORITHMSPREDICTING BANKRUPTCY USING MACHINE LEARNING ALGORITHMS
PREDICTING BANKRUPTCY USING MACHINE LEARNING ALGORITHMS
IJCI JOURNAL
 
A Study On Hybrid System
A Study On Hybrid SystemA Study On Hybrid System
A Study On Hybrid System
Carmen Sanborn
 
Building_a_Readmission_Model_Using_WEKA
Building_a_Readmission_Model_Using_WEKABuilding_a_Readmission_Model_Using_WEKA
Building_a_Readmission_Model_Using_WEKA
Sunil Kakade
 
OSS 2011 Multi-Level Modelling Presentation
OSS 2011 Multi-Level Modelling PresentationOSS 2011 Multi-Level Modelling Presentation
OSS 2011 Multi-Level Modelling Presentation
Timothy Cook
 
Validation and Verification of SYSML Activity Diagrams Using HOARE Logic
Validation and Verification of SYSML Activity Diagrams Using HOARE Logic Validation and Verification of SYSML Activity Diagrams Using HOARE Logic
Validation and Verification of SYSML Activity Diagrams Using HOARE Logic
ijseajournal
 
SBML FOR OPTIMIZING DECISION SUPPORT'S TOOLS
SBML FOR OPTIMIZING DECISION SUPPORT'S TOOLSSBML FOR OPTIMIZING DECISION SUPPORT'S TOOLS
SBML FOR OPTIMIZING DECISION SUPPORT'S TOOLS
csandit
 

Similar to shortstory258 slides.pptx (20)

Simulated Patient Care Pathways
Simulated Patient Care PathwaysSimulated Patient Care Pathways
Simulated Patient Care Pathways
 
BioCreative2023_proceedings_instructions_authors_template.pdf
BioCreative2023_proceedings_instructions_authors_template.pdfBioCreative2023_proceedings_instructions_authors_template.pdf
BioCreative2023_proceedings_instructions_authors_template.pdf
 
Operation research and its application
Operation research and its applicationOperation research and its application
Operation research and its application
 
Controlling informative features for improved accuracy and faster predictions...
Controlling informative features for improved accuracy and faster predictions...Controlling informative features for improved accuracy and faster predictions...
Controlling informative features for improved accuracy and faster predictions...
 
Interpretable Machine Learning_ Techniques for Model Explainability.
Interpretable Machine Learning_ Techniques for Model Explainability.Interpretable Machine Learning_ Techniques for Model Explainability.
Interpretable Machine Learning_ Techniques for Model Explainability.
 
Bio-Inspired Requirements Variability Modeling with use Case
Bio-Inspired Requirements Variability Modeling with use Case Bio-Inspired Requirements Variability Modeling with use Case
Bio-Inspired Requirements Variability Modeling with use Case
 
IRJET - Term based Personalization of Feature Selection of Auto Filling Pa...
IRJET - 	  Term based Personalization of Feature Selection of Auto Filling Pa...IRJET - 	  Term based Personalization of Feature Selection of Auto Filling Pa...
IRJET - Term based Personalization of Feature Selection of Auto Filling Pa...
 
Sbi simulation
Sbi simulationSbi simulation
Sbi simulation
 
BIO-INSPIRED REQUIREMENTS VARIABILITY MODELING WITH USE CASE
BIO-INSPIRED REQUIREMENTS VARIABILITY MODELING WITH USE CASE BIO-INSPIRED REQUIREMENTS VARIABILITY MODELING WITH USE CASE
BIO-INSPIRED REQUIREMENTS VARIABILITY MODELING WITH USE CASE
 
ICU Patient Deterioration Prediction : A Data-Mining Approach
ICU Patient Deterioration Prediction : A Data-Mining ApproachICU Patient Deterioration Prediction : A Data-Mining Approach
ICU Patient Deterioration Prediction : A Data-Mining Approach
 
ICU PATIENT DETERIORATION PREDICTION: A DATA-MINING APPROACH
ICU PATIENT DETERIORATION PREDICTION: A DATA-MINING APPROACHICU PATIENT DETERIORATION PREDICTION: A DATA-MINING APPROACH
ICU PATIENT DETERIORATION PREDICTION: A DATA-MINING APPROACH
 
WP-2013-03-KOLMAT-GL-L
WP-2013-03-KOLMAT-GL-LWP-2013-03-KOLMAT-GL-L
WP-2013-03-KOLMAT-GL-L
 
IMBALANCED DATASET EFFECT ON CNN-BASED CLASSIFIER PERFORMANCE FOR FACE RECOGN...
IMBALANCED DATASET EFFECT ON CNN-BASED CLASSIFIER PERFORMANCE FOR FACE RECOGN...IMBALANCED DATASET EFFECT ON CNN-BASED CLASSIFIER PERFORMANCE FOR FACE RECOGN...
IMBALANCED DATASET EFFECT ON CNN-BASED CLASSIFIER PERFORMANCE FOR FACE RECOGN...
 
Imbalanced Dataset Effect on CNN-Based Classifier Performance for Face Recogn...
Imbalanced Dataset Effect on CNN-Based Classifier Performance for Face Recogn...Imbalanced Dataset Effect on CNN-Based Classifier Performance for Face Recogn...
Imbalanced Dataset Effect on CNN-Based Classifier Performance for Face Recogn...
 
PREDICTING BANKRUPTCY USING MACHINE LEARNING ALGORITHMS
PREDICTING BANKRUPTCY USING MACHINE LEARNING ALGORITHMSPREDICTING BANKRUPTCY USING MACHINE LEARNING ALGORITHMS
PREDICTING BANKRUPTCY USING MACHINE LEARNING ALGORITHMS
 
A Study On Hybrid System
A Study On Hybrid SystemA Study On Hybrid System
A Study On Hybrid System
 
Building_a_Readmission_Model_Using_WEKA
Building_a_Readmission_Model_Using_WEKABuilding_a_Readmission_Model_Using_WEKA
Building_a_Readmission_Model_Using_WEKA
 
OSS 2011 Multi-Level Modelling Presentation
OSS 2011 Multi-Level Modelling PresentationOSS 2011 Multi-Level Modelling Presentation
OSS 2011 Multi-Level Modelling Presentation
 
Validation and Verification of SYSML Activity Diagrams Using HOARE Logic
Validation and Verification of SYSML Activity Diagrams Using HOARE Logic Validation and Verification of SYSML Activity Diagrams Using HOARE Logic
Validation and Verification of SYSML Activity Diagrams Using HOARE Logic
 
SBML FOR OPTIMIZING DECISION SUPPORT'S TOOLS
SBML FOR OPTIMIZING DECISION SUPPORT'S TOOLSSBML FOR OPTIMIZING DECISION SUPPORT'S TOOLS
SBML FOR OPTIMIZING DECISION SUPPORT'S TOOLS
 

Recently uploaded

ScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking ReplicationScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking Replication
ScyllaDB
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Chart Kalyan
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Alex Pruden
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
Ivo Velitchkov
 
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid ResearchHarnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Neo4j
 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
Ajin Abraham
 
Principle of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptxPrinciple of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptx
BibashShahi
 
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
"Scaling RAG Applications to serve millions of users",  Kevin Goedecke"Scaling RAG Applications to serve millions of users",  Kevin Goedecke
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
Fwdays
 
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
Fwdays
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
Day 2 - Intro to UiPath Studio Fundamentals
Day 2 - Intro to UiPath Studio FundamentalsDay 2 - Intro to UiPath Studio Fundamentals
Day 2 - Intro to UiPath Studio Fundamentals
UiPathCommunity
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
Jason Yip
 
High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024
Vadym Kazulkin
 
"What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w..."What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w...
Fwdays
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
LizaNolte
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Edge AI and Vision Alliance
 
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Pitangent Analytics & Technology Solutions Pvt. Ltd
 
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
Fwdays
 
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
zjhamm304
 
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptxPRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
christinelarrosa
 

Recently uploaded (20)

ScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking ReplicationScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking Replication
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
 
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid ResearchHarnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
 
Principle of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptxPrinciple of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptx
 
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
"Scaling RAG Applications to serve millions of users",  Kevin Goedecke"Scaling RAG Applications to serve millions of users",  Kevin Goedecke
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
 
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
Day 2 - Intro to UiPath Studio Fundamentals
Day 2 - Intro to UiPath Studio FundamentalsDay 2 - Intro to UiPath Studio Fundamentals
Day 2 - Intro to UiPath Studio Fundamentals
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
 
High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024
 
"What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w..."What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w...
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
 
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
 
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
 
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
 
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptxPRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
 

shortstory258 slides.pptx

  • 1. THE SHAKY FOUNDATIONS OF CLINICAL FUNCTION MODELS: A SURVEY OF LARGE LANGUAGE MODELS AND FOUNDATION MODELS FOR EMRS. MEDIUM ARTICLE: HTTPS://MEDIUM.COM/@ABDULVAHED.SHAIK/THE- SHAKY-FOUNDATIONS-OF-CLINICAL-FUNCTION-MODELS-D046AA08F737 BASED ON PAPER: HTTPS://ARXIV.ORG/PDF/2303.12961.PDF ABDUL VAHED SHAIK 016452540 SANJOSE STATE UNIVERSITY
  • 2. Two Foundation Models: 1.Clinical Language Models (CLaMs) 2.Foundation models Electronic Medical Records (FEMRs) IN THIS ARTICLE: *Benefits of Clinical FMs *Training Data and Public Availability of models: *Current Evaluation of Clinical FMs: *Improved Evaluation Paradigms for Clinical FMs.
  • 3. Introduction: Foundation models are machine learning models which are easily capable of performing variable tasks on large and huge datasets. FMs have managed to get a lot of attention due to this feature of handling large datasets. It can do text generation, video editing to protein folding and robotics. In case we believe that FMs can help the hospitals and patients in any way, we need to perform some important evaluations, tests to test these assumptions. In this review, we take a walk through Fms and their evaluation regimes assumed clinical value. To clarify on this topic, we reviewed no less than 80 clinical FMs built from the EMR data. We added all the models trained on structured and unstructured data. We are referring to this combination of structured and unstructured EMR data or clinical data.
  • 4. What is Clinical FM? There are mainly two types of foundation models that are built from EMR data. They are Clinical Language Models(CLaMs) and Foundation models for EMRs(FEMRs). Clinical Language Models(CLaMs) : This is the first type of CLaMs, and it is a subtype of large language models(LLMs). It has unique feature of specialization of clinical/biomedical text. Foundation models for Electronic Medical Records(FEMRs) This is the second type of clinical FMs for FEMRs. These models are always trained for all the timeline of all the events in patient’s medical history.The Fig 1.b shows the perfect explanation of FEMRs.A patient’s representation can be useful as input to any type of built models like FEMR.
  • 5. Figure 1. Overview of the inputs and outputs of the two main types of clinical FMs. (a) The inputs and outputs of CLaMs. (b) The inputs and outputs of Foundation models for FEMRs.
  • 6. Benefits of Clinical FMs: 1.Clinical FMs have much better predictive performance. 2.Clinical FMs often require very less labeled data. 3.Clinical FMs enable very simpler and cheaper model deployment. 4.Clinical FMs are much more effective in handling multinomial data. 5.Clinical FMs can also handle novel interfaces for that are useful for human-AI interaction.
  • 7. Training Data and Public Availability of models: CLaMs Training Data: CLaMs are trained on either clinical text or biomedical text. Almost all CLaMs trained on clinical text use a single database: MIMIC-III, which has 2 million notes written. Model Availability: Almost all the CLaMs are publicly accessible via HuggingFace, etc. FEMRs Training data: Most of the FEMRs are trained mostly on small, publicly available EMR datasets or a unique private health system’s EMR database like MIMIC-III. It has less than 40,000 patients. Model Accessibility: FEMRs lack a common process like the HuggingFace for distributing models to the research community.
  • 8. Current Evaluation of Clinical FMs: Clinical FMs are assessed based on the tasks are relatively very easy for evaluation. These provide very limited insight on the FMs being a “categorically different” technology. CLaMs We collected almost every evaluation task which a CLaM was evaluated on its previous original publication. These are evaluated based on standard tasks and standard datasets. FEMRs We collected the real tasks on which each FEMR was evaluated in Figure 3b. Evaluation done based on standard tasks and standard datasets. This is even worse than that of CLaMS. FEMRs lack a huge set of “canonical” evaluations. This makes it highly non feasible to compare the performance of various FEMRs.
  • 9. Figure 2. A depiction of CLaMs way of training, evaluation and publishing.
  • 10. Figure 3. A Depiction of FEMRs and the way they are trained, evaluated and published.
  • 11. Improved Evaluation Paradigms for Clinical FMs. 1.Better Predictive Performance. 2.Less Labeled Data. 3.Simplified Model Deployment. 4.Emergent Clinical Applications. 5.So many emergent clinical applications. 6.Multimodality 7.Novel Human-AI Interfaces.
  • 12. Conclusion This review of 50 CLaMs and 34 FEMRs, shows that most of the clinical FMs are being evaluated on the tasks which give very less information on the advantages of FMs over the traditional ML models. Figure2 and Figure3 show that very less work has been performed to validate if there are any other benefits of FMs. We focused this review mostly on the benefits of clinical FMs with which we can conclude that there are various risks involved and many disadvantages which needs some attention. Similar to traditional ML models , FMs are also open to biases induced by overfitting of the datasets. Keeping all these aside, FMs are really prominent in solving huge range of healthcare complexities.