SlideShare a Scribd company logo
1 of 28
© 2014 IBM Corporation
C203: Big Data and Healthcare: Key
Technologies and Strategies
Turning Big Data Insights into Action through
Advanced Analytics
DATA SUMMIT 2014
Craig Rhinehart
Director, IBM Smarter Care Strategy and
Market Development
© 2014 IBM Corporation 2
C203: Big Data and Healthcare: Key Technologies and Strategies
2:00 pm - 2:45 pm
Turning Big Data Insights into Action through Advanced Analytics
•Healthcare is being disrupted by the collision of a wave of innovation. The
emergence of Big Data, cloud computing, analytics, and social media and
increasing wellness awareness through mobile devices are revolutionizing
patient-centered connected care. This session discusses the new technologies
and approaches that offer promise in terms of better data understanding for
organizations and individuals.
Hashtag: #BigDataNY
Twitter: @CraigRhinehart, @DBTADataSummit
Craig Rhinehart Blog: http://craigrhinehart.com
© 2014 IBM Corporation 3
Please note
IBM’s statements regarding its plans, directions, and intent are subject to change or
withdrawal without notice at IBM’s sole discretion.
Information regarding potential future products is intended to outline our general product
direction and it should not be relied on in making a purchasing decision.
The information mentioned regarding potential future products is not a commitment,
promise, or legal obligation to deliver any material, code or functionality. Information
about potential future products may not be incorporated into any contract. The
development, release, and timing of any future features or functionality described for our
products remains at our sole discretion.
Performance is based on measurements and projections using standard IBM
benchmarks in a controlled environment. The actual throughput or performance
that any user will experience will vary depending upon many factors, including
considerations such as the amount of multiprogramming in the user’s job stream,
the I/O configuration, the storage configuration, and the workload processed.
Therefore, no assurance can be given that an individual user will achieve results
similar to those stated here.
© 2014 IBM Corporation 4
• Time Machine: A Decade of Reversal: An Analysis of
146 Contradicted Medical Practices
• Case Study: Seton Healthcare
• Case Study: University of North Carolina School of
Medicine
• Introduction to Smarter Care
• Questions and Discussion
4
Topics
© 2014 IBM Corporation 5
Who Do You Trust?
A Decade of Reversal:
An Analysis of 146 Contradicted Medical Practices
“40.2% reversed the original standard
of care … and only 38.0% reaffirmed
the original standard of care”
“Amputations or Analytics …”
Craig Rhinehart Blog
© 2014 IBM Corporation 6
(Billion) Wasted on missed
opportunities along with
unnecessary, error-prone and
inefficiently delivered services 3
$585B
US physicians still reliant on
paper based medical
records systems 4
45%
Hours each day an average
family physician spends on
direct patient care 3
8.0
Hours required to meet the
patient care guidelines each
day 3
21.7
1 World Health Statistics 2011 from World Health Organization
2 The World Health Report 2000 – Health Systems: Improving Performance from World Health Organization
3 Best Care at Lower Cost: The Path to Continuously Learning Health Care in America from Institute of Medicine / National Academy of Sciences
4 Physician Adoption of Electronic Health Record Systems: United States, 2011 from National Center for Health Statistics
US rank in Healthcare
spending 1
1st
US rank in quality of care
delivered 2
37th
80%90% of the world’s data is unstructuredof the world’s data was
created in the last two years
An Ocean of Unused
Data
Healthcare Transformation: A Work in Progress
© 2014 IBM Corporation 7
Unstructured data is messy but filled with key
medical facts
Medications, diseases, symptoms, non-symptoms,
lab measurements, social history, family history …
and much more
© 2014 IBM Corporation 8
20% of people generate
80% of costs
Health care spending
Health status
Healthy
Low Risk
High RiskAt Risk
Active
Disease
Early
Clinical
Symptoms
Disease and cost-of-care progression
Time
Early intervention opportunities identification
Early intervention opportunities identification
70% of US deaths from
chronic diseases
© 2014 IBM Corporation 9
Seton Healthcare Family
Reducing CHF readmission to improve care
Business Challenge
Seton Healthcare strives to reduce the occurrence of high
cost Congestive Heart Failure (CHF) readmissions by
proactively identifying patients likely to be readmitted on an
emergent basis.
What’s Smart?
IBM Content and Predictive Analytics for Healthcare
solution will help to better target and understand high-risk
CHF patients for care management programs by:
Smarter Business Outcomes
• Seton will be able to proactively target care management
and reduce re-admission of CHF patients.
• Teaming unstructured content with predictive analytics,
Seton will be able to identify patients likely for re-
admission and introduce early interventions to reduce
cost, mortality rates, and improved patient quality of life.
IBM solution
• IBM Content and
Predictive Analytics
for Healthcare
• IBM Cognos Business
Intelligence
• IBM BAO solution
services
• Utilizing natural language processing to extract key elements
from unstructured History and Physical, Discharge Summaries,
Echocardiogram Reports, and Consult Notes
• Leveraging predictive models that have demonstrated high
positive predictive value against extracted elements of
structured and unstructured data
• Providing an interface through which providers can intuitively
navigate, interpret and take action
“IBM Content and Predictive Analytics for Healthcare uses the same type of natural language processing as IBM Watson,
enabling us to leverage information in new ways not possible before. We can access an integrated view of relevant clinical
and operational information to drive more informed decision making and optimize patient and operational outcomes.”
Charles J. Barnett, FACHE, President/Chief Executive Officer, Seton Healthcare Family
Featured on
© 2014 IBM Corporation 10
The Data We Thought Would Be Useful … Wasn’t
• Structured data not available, not accurate enough, without the unstructured data - which was more trustworthy
What We Thought Was Causing 30 Day Readmissions … Wasn’t
• 113 possible candidate predictors expanded and changed after mining the data for hidden insights
New Hidden Indicators Emerged … Readmissions is a Highly Predictive Model
• 18 accurate indicators or predictors (see next slide)
Predictor Analysis % Encounters
Structured Data
% Encounters
Unstructured Data
Ejection Fraction (LVEF) 2% 74%
Smoking Indicator 35%
(65% Accurate)
81%
(95% Accurate)
Living Arrangements <1% 73%
(100% Accurate)
Drug and Alcohol Abuse 16% 81%
Assisted Living 0% 13%
What Really Causes Readmissions at Seton
Key Findings from Seton’s Data
97% at 80th
percentile
49% at 20th
percentile
© 2014 IBM Corporation 11
What Really Causes Readmissions at Seton
Top 18 Indicators
1. Jugular Venous Distention Indicator
2. Paid by Medicaid Indicator
3. Immunity Disorder Disease Indicator
4. Cardiac Rehab Admit Diagnosis with CHF Indicator
5. Lack of Emotion Support Indicator
6. Self COPD Moderate Limit Health History Indicator
7. With Genitourinary System and Endocrine Disorders
8. Heart Failure History
9. High BNP Indicator
10. Low Hemoglobin Indicator
11. Low Sodium Level Indicator
12. Assisted Living
13. High Cholesterol History
14. Presence of Blood Diseases in Diagnosis History
15. High Blood Pressure Health History
16. Self Alcohol / Drug Use Indicator
17. Heart Attack History
18. Heart Disease History
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
0 1 2 3 4 5 6
RankingofStrengthofModelVariable
Projected OddsRa o
18 17 16 15 14 13 12 11 10
9 8 7 6 5 4 3 2 1
New Insights Uncovered by Combining Content and Predictive Analytics
• Top indicator JVDI not on the original list of 113 - as well as several others
• Assisted Living and Drug and Alcohol Abuse emerged as key predictors - only found in
unstructured data
• LVEF and Smoking are significant indicators of CHF but not readmissions
• A combination of actionable and non-actionable factors cause readmissions
© 2014 IBM Corporation 12
The Impact of Readmissions at Seton
CHF Patient X – What Happened? Admit / Readmission
30-Day Readmission
98% 98% 96% 95% 96% 100%
Apr-18-2009 May-12-2009 May-20-2009 Oct-11-2009 Nov-24-2009 Dec-20-2009
8 days24 days 144 days 44 days 26 days
Individual Patient Data at Each Encounter (Patient X @ Dec 20, 2009)
Patient X was hospitalized 6 times over an 8 month period. The same basic
information was available at each encounter and Patient X’s readmission
prediction score never dropped below 95% (out of possible 100%)
Patient Population Monitoring
Clinical and Operational Data
© 2014 IBM Corporation 13
Overview
• Improve collection of Physician Quality Reporting
System (PQRS) measures
• Prepare for meaningful use (Stage 3) and improve test
result follow-up
• Avoid Centers for Medicare and Medicaid Services
(CMS) readmission penalties
© 2014 IBM Corporation 14
Physician Quality Reporting System (PQRS)
measures
PQRS is a reporting program that uses a
combination of incentive payments and payment
adjustments to promote reporting of quality
information by eligible professionals (EP)
• Services furnished for Medicare Part B
PQRS measures
• Breast cancer screening (mammograms)
• Colorectal cancer screening
• Pneumovax
© 2014 IBM Corporation 15
Problem
Mechanism for capturing procedures
performed at outside institutions is
unreliable:
• Mammogram
• Colo-rectal cancer (CRC) screening
• Pneumovax
PQRS measures are typically captured
using structured Electronic Medical
Record (EMR) data
© 2014 IBM Corporation 16
Health maintenance in electronic medical record
© 2014 IBM Corporation 17
Free-text note in patient’s EMR
Not captured in
structured data
© 2014 IBM Corporation 18
Physician Quality Reporting System (PQRS)
measures
UNC Healthcare utilized IBM Content
Analytics (ICA) to analyze free-text clinical
documents to improve the accuracy of our 2012
PQRS measures
We used ICA to analyze clinical notes for
those patients whose structured EMR data did
not indicate compliance with PQRS measure
requirements
• Mammogram, CRC screening, Pneumovax
© 2014 IBM Corporation 19
Results
142
155
236
49
27
247
Mammogram
(297)
CRC screening
(285)
Pneumovax
(271)NLP Capture of PQRS
Measure
0
50
100
150
200
350
300
250
Numberofpatientsoutofcompliance
(structureddata)
© 2014 IBM Corporation 20
Results
60
53
55
61
Mammogram CRC screening Pneumovax
Structured data
48
50
52
54
56
64
62
58
Percentofpatientsmeetingrequirements
60
Structured data + NLP
53
63
© 2014 IBM Corporation 21
Follow-up of abnormal test results documented in
free-text reports
Mammograms | Colonoscopies | Pap tests | Chest x-rays
© 2014 IBM Corporation 22
Breast cancer and mammograms
• Second deadliest cancer in women
• Screening mammograms can
reduce mortality
28%of women requiring short-term follow-up for
abnormal mammography results do not receive
recommended care
© 2014 IBM Corporation 23
Mammography reports
© 2014 IBM Corporation 24
Performance of NLP for accurately identifying
mammography results
Characteristics Value
Precision 98%
Recall 100%
F Measure 99%
• Precision = positive predictive value
If NLP reports a result, the probability
that the result is correct
• Recall = sensitivity
if there is a result in the mammography
report, the probability that NLP identifies
and accurately classifies the result
• F-measure = summary measure of
NLP performance
Revised NLP model has 100% performance for all characteristics
© 2014 IBM Corporation 25
Structured Data is Not Enough
 Unstructured data significantly increases the
richness and accuracy of analysis and decision
making … including paper / faxes
Today’s Care Guidelines Only Get You So Far
 Not granular enough to deliver on the promise of
personalized medicine with data driven insights 1,2
Manual Processes and Traditional Workflow
Approaches Don’t Work
 Process complexity increases with disease
complexity … changing conditions require process
adaptability 3
© 2012 IBM Corporation
Prediction Results of Knowledge-driven Features plus Data-driven
Features
!AUC significantly improves as complementary data driven risk factors are
added into existing knowledge based risk factors.
!A significant AUC increase occurs when we add first 50 data driven features
! " # # $! %&# $
! %# # $
! &# $
' (($) * + , (- . / - $
0- ' 12 3- 4$! . 5' 6 - 1- 4$
! 7 8 9 - 31- * 45+ * $
: ; < $
# =&$
# =&&$
# => $
# => &$
# =? $
# =? &$
# =@$
# $ %# # $ " # # $ A # # $ B # # $ &# # $ > # # $
!"#$
% &' ( ) *$+ , $, ) - . &*) / $
Knowledge and
Guidelines
Data Driven
Insights
1. Dijun Luo, Fie Wang, Jimeng Sun, Marianthi Markatou, Jianying Hu,Shahram Ebadollahi, SOR: ScalableOrthogonal Regression for Low-
Redundancy Feature Selection and its Healthcare Applications. SDM’12
2. Jimeng Sun, Jianying Hu, Dijun Luo, Marianthi Markatou, Fei Wang, Shahram Edabollahi, Steven E. Steinhubl, Zahra Daar, Walter F. Stewart.
Combining Knowledge and Data Driven Insights for Identifying Risk Factors using Electronic Health Records. Under submission at AMIA’12
3. Blind Surgeon Metaphor Problem - W.M.P. van der Aalst, M. Weske, and D. Grünbauer. Case Handling: A New Paradigm for Business Process
Support. Data and Knowledge Engineering, 53(2):129-162, 2005
What Have We Learned So Far?
© 2014 IBM Corporation 26
A Smarter approach to delivering better outcomes
• Build longitudinal “data driven”
evidence based population insights
• Uncover hidden intervention
opportunities
• Proactively deliver accountable and
personalized care in a patient
centered model
• Collaborate across caregivers to focus
on high cost, high need patients
• Prevent at-risk patients from
progressing to high cost, high need
IBM Strategy to Support Care Coordination:
© 2014 IBM Corporation 27
The path forward
IBM Smarter Care uncovers valuable insights into lifestyle choices, social determinants,
and clinical factors enabling holistic and individualized care to optimize outcomes and
lower costs
Social
determinants such as where one is born,
grows, lives, works and ages have direct
impact on an individual’s overall health and
well being
Lifestyle
choices have direct impact on an
individual’s mental and physical wellness
Clinical
factors such as specific medical symptoms,
history, medications, diagnoses, etc are
indicators of an individual’s health
© 2014 IBM Corporation 28
© 2013 IBM Corporation
28
Find out more about IBM Smarter Care
http://www.ibm.com/smarterplanet/us/en/smarter_care/overview/
Visit my blog or follow me on Twitter
http://craigrhinehart.com
@CraigRhinehart
Craig Rhinehart
Director, IBM Smarter Care Strategy and Market Development
craigrhinehart@us.ibm.com
Thank You

More Related Content

What's hot

Tapping into the Potential of Natural Language Processing in Healthcare
Tapping into the Potential of Natural Language Processing in HealthcareTapping into the Potential of Natural Language Processing in Healthcare
Tapping into the Potential of Natural Language Processing in Healthcare
Health Catalyst
 
On Medicaid claims data analysis
On Medicaid claims data analysisOn Medicaid claims data analysis
On Medicaid claims data analysis
Elspeth Slayter
 
Machine Learning Misconceptions
Machine Learning MisconceptionsMachine Learning Misconceptions
Machine Learning Misconceptions
Health Catalyst
 

What's hot (20)

Use of data analytics in health care
Use of data analytics in health careUse of data analytics in health care
Use of data analytics in health care
 
The Why And How Of Machine Learning And AI: An Implementation Guide For Healt...
The Why And How Of Machine Learning And AI: An Implementation Guide For Healt...The Why And How Of Machine Learning And AI: An Implementation Guide For Healt...
The Why And How Of Machine Learning And AI: An Implementation Guide For Healt...
 
Tapping into the Potential of Natural Language Processing in Healthcare
Tapping into the Potential of Natural Language Processing in HealthcareTapping into the Potential of Natural Language Processing in Healthcare
Tapping into the Potential of Natural Language Processing in Healthcare
 
The Changing Role of Healthcare Data Analysts
The Changing Role of Healthcare Data AnalystsThe Changing Role of Healthcare Data Analysts
The Changing Role of Healthcare Data Analysts
 
Ai startups in healthcare By Dr.Mahboob Khan
Ai startups in healthcare By Dr.Mahboob Khan Ai startups in healthcare By Dr.Mahboob Khan
Ai startups in healthcare By Dr.Mahboob Khan
 
On Medicaid claims data analysis
On Medicaid claims data analysisOn Medicaid claims data analysis
On Medicaid claims data analysis
 
Machine Learning Misconceptions
Machine Learning MisconceptionsMachine Learning Misconceptions
Machine Learning Misconceptions
 
7 keys to open doors at CVS
7 keys to open doors at CVS7 keys to open doors at CVS
7 keys to open doors at CVS
 
Four Effective Opioid Interventions for Healthcare Leaders
Four Effective Opioid Interventions for Healthcare LeadersFour Effective Opioid Interventions for Healthcare Leaders
Four Effective Opioid Interventions for Healthcare Leaders
 
Four Steps to Effective Opportunity Analysis
Four Steps to Effective Opportunity AnalysisFour Steps to Effective Opportunity Analysis
Four Steps to Effective Opportunity Analysis
 
Predictive analytics in health insurance
Predictive analytics in health insurancePredictive analytics in health insurance
Predictive analytics in health insurance
 
Cloud-Based Open-Platform Data Solutions: The Best Way to Meet Today’s Growin...
Cloud-Based Open-Platform Data Solutions: The Best Way to Meet Today’s Growin...Cloud-Based Open-Platform Data Solutions: The Best Way to Meet Today’s Growin...
Cloud-Based Open-Platform Data Solutions: The Best Way to Meet Today’s Growin...
 
How Addressing Mental Health Can Improve Chronic Disease Outcomes and Cut Costs
How Addressing Mental Health Can Improve Chronic Disease Outcomes and Cut CostsHow Addressing Mental Health Can Improve Chronic Disease Outcomes and Cut Costs
How Addressing Mental Health Can Improve Chronic Disease Outcomes and Cut Costs
 
Truven Health Overview
Truven Health OverviewTruven Health Overview
Truven Health Overview
 
Big data in health care
Big data in health careBig data in health care
Big data in health care
 
Healthcare Interoperability: New Tactics and Technology
Healthcare Interoperability: New Tactics and TechnologyHealthcare Interoperability: New Tactics and Technology
Healthcare Interoperability: New Tactics and Technology
 
What Healthcare Analysts Can Learn About Data Analytics from the World of Sur...
What Healthcare Analysts Can Learn About Data Analytics from the World of Sur...What Healthcare Analysts Can Learn About Data Analytics from the World of Sur...
What Healthcare Analysts Can Learn About Data Analytics from the World of Sur...
 
Why Data-Driven Healthcare Is the Best Defense Against COVID-19
Why Data-Driven Healthcare Is the Best Defense Against COVID-19Why Data-Driven Healthcare Is the Best Defense Against COVID-19
Why Data-Driven Healthcare Is the Best Defense Against COVID-19
 
Lean Healthcare: 6 Methodologies for Improvement from Dr. Brent James
Lean Healthcare: 6 Methodologies for Improvement from Dr. Brent JamesLean Healthcare: 6 Methodologies for Improvement from Dr. Brent James
Lean Healthcare: 6 Methodologies for Improvement from Dr. Brent James
 
Demystifying Text Analytics and NLP in Healthcare
Demystifying Text Analytics and NLP in HealthcareDemystifying Text Analytics and NLP in Healthcare
Demystifying Text Analytics and NLP in Healthcare
 

Viewers also liked

Viewers also liked (9)

Healthcare 2.0: The Age of Analytics
Healthcare 2.0: The Age of AnalyticsHealthcare 2.0: The Age of Analytics
Healthcare 2.0: The Age of Analytics
 
Healthcare Analytics Adoption Model
Healthcare Analytics Adoption ModelHealthcare Analytics Adoption Model
Healthcare Analytics Adoption Model
 
AWS re:Invent 2016: Case Study: Data-Heavy Healthcare: UPMCe’s Transformative...
AWS re:Invent 2016: Case Study: Data-Heavy Healthcare: UPMCe’s Transformative...AWS re:Invent 2016: Case Study: Data-Heavy Healthcare: UPMCe’s Transformative...
AWS re:Invent 2016: Case Study: Data-Heavy Healthcare: UPMCe’s Transformative...
 
Intrapreneurship: Tackling The Challenges of Bringing New Innovation to Market
Intrapreneurship: Tackling The Challenges of Bringing New Innovation to MarketIntrapreneurship: Tackling The Challenges of Bringing New Innovation to Market
Intrapreneurship: Tackling The Challenges of Bringing New Innovation to Market
 
AWS re:Invent 2016| HLC301 | Data Science and Healthcare: Running Large Scale...
AWS re:Invent 2016| HLC301 | Data Science and Healthcare: Running Large Scale...AWS re:Invent 2016| HLC301 | Data Science and Healthcare: Running Large Scale...
AWS re:Invent 2016| HLC301 | Data Science and Healthcare: Running Large Scale...
 
AWS re:Invent 2016: How to Build a Big Data Analytics Data Lake (LFS303)
AWS re:Invent 2016: How to Build a Big Data Analytics Data Lake (LFS303)AWS re:Invent 2016: How to Build a Big Data Analytics Data Lake (LFS303)
AWS re:Invent 2016: How to Build a Big Data Analytics Data Lake (LFS303)
 
Big data architectures and the data lake
Big data architectures and the data lakeBig data architectures and the data lake
Big data architectures and the data lake
 
SlideShare 101
SlideShare 101SlideShare 101
SlideShare 101
 
Visual Design with Data
Visual Design with DataVisual Design with Data
Visual Design with Data
 

Similar to Turning Big Data Insights into Action through Advanced Analytics

TeraCrunch: Transforming Organizations with Machien Learning and Gen-AI Solut...
TeraCrunch: Transforming Organizations with Machien Learning and Gen-AI Solut...TeraCrunch: Transforming Organizations with Machien Learning and Gen-AI Solut...
TeraCrunch: Transforming Organizations with Machien Learning and Gen-AI Solut...
KC Digital Drive
 
Amy walker aami_%202011(7)
Amy walker aami_%202011(7)Amy walker aami_%202011(7)
Amy walker aami_%202011(7)
Amy Stowers
 
BRG Albany HFMA 4.23.15
BRG Albany HFMA 4.23.15BRG Albany HFMA 4.23.15
BRG Albany HFMA 4.23.15
Mark Driscoll
 
Healthcare Management Powerpoint Presentation Slides
Healthcare Management Powerpoint Presentation SlidesHealthcare Management Powerpoint Presentation Slides
Healthcare Management Powerpoint Presentation Slides
SlideTeam
 
McGrath Health Data Analyst SXSW
McGrath Health Data Analyst SXSWMcGrath Health Data Analyst SXSW
McGrath Health Data Analyst SXSW
Robert McGrath
 
Healthcare Management PowerPoint Presentation Slides
Healthcare Management PowerPoint Presentation SlidesHealthcare Management PowerPoint Presentation Slides
Healthcare Management PowerPoint Presentation Slides
SlideTeam
 
Clinical Decision Support: Driving the Last Mile
Clinical Decision Support: Driving the Last MileClinical Decision Support: Driving the Last Mile
Clinical Decision Support: Driving the Last Mile
Health Catalyst
 

Similar to Turning Big Data Insights into Action through Advanced Analytics (20)

Predictive Health Population Analytics
Predictive Health Population AnalyticsPredictive Health Population Analytics
Predictive Health Population Analytics
 
2016 IBM Interconnect - medical devices transformation
2016 IBM Interconnect  - medical devices transformation2016 IBM Interconnect  - medical devices transformation
2016 IBM Interconnect - medical devices transformation
 
Miles Snowden, MD - Prevention, Wellness & Outcomes from a Payer Prospective
Miles Snowden, MD - Prevention, Wellness & Outcomes from a Payer ProspectiveMiles Snowden, MD - Prevention, Wellness & Outcomes from a Payer Prospective
Miles Snowden, MD - Prevention, Wellness & Outcomes from a Payer Prospective
 
Aami hitech mu impact on the future on HC IT
Aami hitech mu impact on the future on HC ITAami hitech mu impact on the future on HC IT
Aami hitech mu impact on the future on HC IT
 
Transforming Healthcare through Data & Analytics
Transforming Healthcare through Data & AnalyticsTransforming Healthcare through Data & Analytics
Transforming Healthcare through Data & Analytics
 
Shapingthenewhealthcaresystem keynoteaddressbyonurtorusoglu-171031161749
Shapingthenewhealthcaresystem keynoteaddressbyonurtorusoglu-171031161749Shapingthenewhealthcaresystem keynoteaddressbyonurtorusoglu-171031161749
Shapingthenewhealthcaresystem keynoteaddressbyonurtorusoglu-171031161749
 
Your cognitive future: How next-gen computing changes the way we live and work
Your cognitive future: How next-gen computing changes the way we live and workYour cognitive future: How next-gen computing changes the way we live and work
Your cognitive future: How next-gen computing changes the way we live and work
 
Big Data Provides Opportunities, Challenges and a Better Future in Health and...
Big Data Provides Opportunities, Challenges and a Better Future in Health and...Big Data Provides Opportunities, Challenges and a Better Future in Health and...
Big Data Provides Opportunities, Challenges and a Better Future in Health and...
 
TeraCrunch: Transforming Organizations with Machien Learning and Gen-AI Solut...
TeraCrunch: Transforming Organizations with Machien Learning and Gen-AI Solut...TeraCrunch: Transforming Organizations with Machien Learning and Gen-AI Solut...
TeraCrunch: Transforming Organizations with Machien Learning and Gen-AI Solut...
 
Paul Grundy, Better Care, Reducing Costs, Improving Service Patient Centered ...
Paul Grundy, Better Care, Reducing Costs, Improving Service Patient Centered ...Paul Grundy, Better Care, Reducing Costs, Improving Service Patient Centered ...
Paul Grundy, Better Care, Reducing Costs, Improving Service Patient Centered ...
 
Healthcare IT Services Insights - January 2016
Healthcare IT Services Insights - January 2016Healthcare IT Services Insights - January 2016
Healthcare IT Services Insights - January 2016
 
Amy walker aami_%202011(7)
Amy walker aami_%202011(7)Amy walker aami_%202011(7)
Amy walker aami_%202011(7)
 
How to Use Data to Improve Patient Safety: Part 2
How to Use Data to Improve Patient Safety: Part 2How to Use Data to Improve Patient Safety: Part 2
How to Use Data to Improve Patient Safety: Part 2
 
The Philosophy, Psychology, and Technology of Data in Healthcare
The Philosophy, Psychology, and  Technology of Data in HealthcareThe Philosophy, Psychology, and  Technology of Data in Healthcare
The Philosophy, Psychology, and Technology of Data in Healthcare
 
BRG Albany HFMA 4.23.15
BRG Albany HFMA 4.23.15BRG Albany HFMA 4.23.15
BRG Albany HFMA 4.23.15
 
Healthcare Management Powerpoint Presentation Slides
Healthcare Management Powerpoint Presentation SlidesHealthcare Management Powerpoint Presentation Slides
Healthcare Management Powerpoint Presentation Slides
 
2019 Healthcare Trends
2019 Healthcare Trends 2019 Healthcare Trends
2019 Healthcare Trends
 
McGrath Health Data Analyst SXSW
McGrath Health Data Analyst SXSWMcGrath Health Data Analyst SXSW
McGrath Health Data Analyst SXSW
 
Healthcare Management PowerPoint Presentation Slides
Healthcare Management PowerPoint Presentation SlidesHealthcare Management PowerPoint Presentation Slides
Healthcare Management PowerPoint Presentation Slides
 
Clinical Decision Support: Driving the Last Mile
Clinical Decision Support: Driving the Last MileClinical Decision Support: Driving the Last Mile
Clinical Decision Support: Driving the Last Mile
 

Recently uploaded

Probability Grade 10 Third Quarter Lessons
Probability Grade 10 Third Quarter LessonsProbability Grade 10 Third Quarter Lessons
Probability Grade 10 Third Quarter Lessons
JoseMangaJr1
 
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts ServiceCall Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
9953056974 Low Rate Call Girls In Saket, Delhi NCR
 
Abortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Doha Qatar (+966572737505 ! Get CytotecAbortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Riyadh +966572737505 get cytotec
 
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICECHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
9953056974 Low Rate Call Girls In Saket, Delhi NCR
 
FESE Capital Markets Fact Sheet 2024 Q1.pdf
FESE Capital Markets Fact Sheet 2024 Q1.pdfFESE Capital Markets Fact Sheet 2024 Q1.pdf
FESE Capital Markets Fact Sheet 2024 Q1.pdf
MarinCaroMartnezBerg
 

Recently uploaded (20)

Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 nightCheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
 
Edukaciniai dropshipping via API with DroFx
Edukaciniai dropshipping via API with DroFxEdukaciniai dropshipping via API with DroFx
Edukaciniai dropshipping via API with DroFx
 
(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7
(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7
(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7
 
Probability Grade 10 Third Quarter Lessons
Probability Grade 10 Third Quarter LessonsProbability Grade 10 Third Quarter Lessons
Probability Grade 10 Third Quarter Lessons
 
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts ServiceCall Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
 
Discover Why Less is More in B2B Research
Discover Why Less is More in B2B ResearchDiscover Why Less is More in B2B Research
Discover Why Less is More in B2B Research
 
Abortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Doha Qatar (+966572737505 ! Get CytotecAbortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Doha Qatar (+966572737505 ! Get Cytotec
 
Halmar dropshipping via API with DroFx
Halmar  dropshipping  via API with DroFxHalmar  dropshipping  via API with DroFx
Halmar dropshipping via API with DroFx
 
VidaXL dropshipping via API with DroFx.pptx
VidaXL dropshipping via API with DroFx.pptxVidaXL dropshipping via API with DroFx.pptx
VidaXL dropshipping via API with DroFx.pptx
 
Midocean dropshipping via API with DroFx
Midocean dropshipping via API with DroFxMidocean dropshipping via API with DroFx
Midocean dropshipping via API with DroFx
 
Anomaly detection and data imputation within time series
Anomaly detection and data imputation within time seriesAnomaly detection and data imputation within time series
Anomaly detection and data imputation within time series
 
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICECHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
 
Week-01-2.ppt BBB human Computer interaction
Week-01-2.ppt BBB human Computer interactionWeek-01-2.ppt BBB human Computer interaction
Week-01-2.ppt BBB human Computer interaction
 
BabyOno dropshipping via API with DroFx.pptx
BabyOno dropshipping via API with DroFx.pptxBabyOno dropshipping via API with DroFx.pptx
BabyOno dropshipping via API with DroFx.pptx
 
Generative AI on Enterprise Cloud with NiFi and Milvus
Generative AI on Enterprise Cloud with NiFi and MilvusGenerative AI on Enterprise Cloud with NiFi and Milvus
Generative AI on Enterprise Cloud with NiFi and Milvus
 
CebaBaby dropshipping via API with DroFX.pptx
CebaBaby dropshipping via API with DroFX.pptxCebaBaby dropshipping via API with DroFX.pptx
CebaBaby dropshipping via API with DroFX.pptx
 
FESE Capital Markets Fact Sheet 2024 Q1.pdf
FESE Capital Markets Fact Sheet 2024 Q1.pdfFESE Capital Markets Fact Sheet 2024 Q1.pdf
FESE Capital Markets Fact Sheet 2024 Q1.pdf
 
ALSO dropshipping via API with DroFx.pptx
ALSO dropshipping via API with DroFx.pptxALSO dropshipping via API with DroFx.pptx
ALSO dropshipping via API with DroFx.pptx
 
ELKO dropshipping via API with DroFx.pptx
ELKO dropshipping via API with DroFx.pptxELKO dropshipping via API with DroFx.pptx
ELKO dropshipping via API with DroFx.pptx
 
Mature dropshipping via API with DroFx.pptx
Mature dropshipping via API with DroFx.pptxMature dropshipping via API with DroFx.pptx
Mature dropshipping via API with DroFx.pptx
 

Turning Big Data Insights into Action through Advanced Analytics

  • 1. © 2014 IBM Corporation C203: Big Data and Healthcare: Key Technologies and Strategies Turning Big Data Insights into Action through Advanced Analytics DATA SUMMIT 2014 Craig Rhinehart Director, IBM Smarter Care Strategy and Market Development
  • 2. © 2014 IBM Corporation 2 C203: Big Data and Healthcare: Key Technologies and Strategies 2:00 pm - 2:45 pm Turning Big Data Insights into Action through Advanced Analytics •Healthcare is being disrupted by the collision of a wave of innovation. The emergence of Big Data, cloud computing, analytics, and social media and increasing wellness awareness through mobile devices are revolutionizing patient-centered connected care. This session discusses the new technologies and approaches that offer promise in terms of better data understanding for organizations and individuals. Hashtag: #BigDataNY Twitter: @CraigRhinehart, @DBTADataSummit Craig Rhinehart Blog: http://craigrhinehart.com
  • 3. © 2014 IBM Corporation 3 Please note IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
  • 4. © 2014 IBM Corporation 4 • Time Machine: A Decade of Reversal: An Analysis of 146 Contradicted Medical Practices • Case Study: Seton Healthcare • Case Study: University of North Carolina School of Medicine • Introduction to Smarter Care • Questions and Discussion 4 Topics
  • 5. © 2014 IBM Corporation 5 Who Do You Trust? A Decade of Reversal: An Analysis of 146 Contradicted Medical Practices “40.2% reversed the original standard of care … and only 38.0% reaffirmed the original standard of care” “Amputations or Analytics …” Craig Rhinehart Blog
  • 6. © 2014 IBM Corporation 6 (Billion) Wasted on missed opportunities along with unnecessary, error-prone and inefficiently delivered services 3 $585B US physicians still reliant on paper based medical records systems 4 45% Hours each day an average family physician spends on direct patient care 3 8.0 Hours required to meet the patient care guidelines each day 3 21.7 1 World Health Statistics 2011 from World Health Organization 2 The World Health Report 2000 – Health Systems: Improving Performance from World Health Organization 3 Best Care at Lower Cost: The Path to Continuously Learning Health Care in America from Institute of Medicine / National Academy of Sciences 4 Physician Adoption of Electronic Health Record Systems: United States, 2011 from National Center for Health Statistics US rank in Healthcare spending 1 1st US rank in quality of care delivered 2 37th 80%90% of the world’s data is unstructuredof the world’s data was created in the last two years An Ocean of Unused Data Healthcare Transformation: A Work in Progress
  • 7. © 2014 IBM Corporation 7 Unstructured data is messy but filled with key medical facts Medications, diseases, symptoms, non-symptoms, lab measurements, social history, family history … and much more
  • 8. © 2014 IBM Corporation 8 20% of people generate 80% of costs Health care spending Health status Healthy Low Risk High RiskAt Risk Active Disease Early Clinical Symptoms Disease and cost-of-care progression Time Early intervention opportunities identification Early intervention opportunities identification 70% of US deaths from chronic diseases
  • 9. © 2014 IBM Corporation 9 Seton Healthcare Family Reducing CHF readmission to improve care Business Challenge Seton Healthcare strives to reduce the occurrence of high cost Congestive Heart Failure (CHF) readmissions by proactively identifying patients likely to be readmitted on an emergent basis. What’s Smart? IBM Content and Predictive Analytics for Healthcare solution will help to better target and understand high-risk CHF patients for care management programs by: Smarter Business Outcomes • Seton will be able to proactively target care management and reduce re-admission of CHF patients. • Teaming unstructured content with predictive analytics, Seton will be able to identify patients likely for re- admission and introduce early interventions to reduce cost, mortality rates, and improved patient quality of life. IBM solution • IBM Content and Predictive Analytics for Healthcare • IBM Cognos Business Intelligence • IBM BAO solution services • Utilizing natural language processing to extract key elements from unstructured History and Physical, Discharge Summaries, Echocardiogram Reports, and Consult Notes • Leveraging predictive models that have demonstrated high positive predictive value against extracted elements of structured and unstructured data • Providing an interface through which providers can intuitively navigate, interpret and take action “IBM Content and Predictive Analytics for Healthcare uses the same type of natural language processing as IBM Watson, enabling us to leverage information in new ways not possible before. We can access an integrated view of relevant clinical and operational information to drive more informed decision making and optimize patient and operational outcomes.” Charles J. Barnett, FACHE, President/Chief Executive Officer, Seton Healthcare Family Featured on
  • 10. © 2014 IBM Corporation 10 The Data We Thought Would Be Useful … Wasn’t • Structured data not available, not accurate enough, without the unstructured data - which was more trustworthy What We Thought Was Causing 30 Day Readmissions … Wasn’t • 113 possible candidate predictors expanded and changed after mining the data for hidden insights New Hidden Indicators Emerged … Readmissions is a Highly Predictive Model • 18 accurate indicators or predictors (see next slide) Predictor Analysis % Encounters Structured Data % Encounters Unstructured Data Ejection Fraction (LVEF) 2% 74% Smoking Indicator 35% (65% Accurate) 81% (95% Accurate) Living Arrangements <1% 73% (100% Accurate) Drug and Alcohol Abuse 16% 81% Assisted Living 0% 13% What Really Causes Readmissions at Seton Key Findings from Seton’s Data 97% at 80th percentile 49% at 20th percentile
  • 11. © 2014 IBM Corporation 11 What Really Causes Readmissions at Seton Top 18 Indicators 1. Jugular Venous Distention Indicator 2. Paid by Medicaid Indicator 3. Immunity Disorder Disease Indicator 4. Cardiac Rehab Admit Diagnosis with CHF Indicator 5. Lack of Emotion Support Indicator 6. Self COPD Moderate Limit Health History Indicator 7. With Genitourinary System and Endocrine Disorders 8. Heart Failure History 9. High BNP Indicator 10. Low Hemoglobin Indicator 11. Low Sodium Level Indicator 12. Assisted Living 13. High Cholesterol History 14. Presence of Blood Diseases in Diagnosis History 15. High Blood Pressure Health History 16. Self Alcohol / Drug Use Indicator 17. Heart Attack History 18. Heart Disease History 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 0 1 2 3 4 5 6 RankingofStrengthofModelVariable Projected OddsRa o 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 New Insights Uncovered by Combining Content and Predictive Analytics • Top indicator JVDI not on the original list of 113 - as well as several others • Assisted Living and Drug and Alcohol Abuse emerged as key predictors - only found in unstructured data • LVEF and Smoking are significant indicators of CHF but not readmissions • A combination of actionable and non-actionable factors cause readmissions
  • 12. © 2014 IBM Corporation 12 The Impact of Readmissions at Seton CHF Patient X – What Happened? Admit / Readmission 30-Day Readmission 98% 98% 96% 95% 96% 100% Apr-18-2009 May-12-2009 May-20-2009 Oct-11-2009 Nov-24-2009 Dec-20-2009 8 days24 days 144 days 44 days 26 days Individual Patient Data at Each Encounter (Patient X @ Dec 20, 2009) Patient X was hospitalized 6 times over an 8 month period. The same basic information was available at each encounter and Patient X’s readmission prediction score never dropped below 95% (out of possible 100%) Patient Population Monitoring Clinical and Operational Data
  • 13. © 2014 IBM Corporation 13 Overview • Improve collection of Physician Quality Reporting System (PQRS) measures • Prepare for meaningful use (Stage 3) and improve test result follow-up • Avoid Centers for Medicare and Medicaid Services (CMS) readmission penalties
  • 14. © 2014 IBM Corporation 14 Physician Quality Reporting System (PQRS) measures PQRS is a reporting program that uses a combination of incentive payments and payment adjustments to promote reporting of quality information by eligible professionals (EP) • Services furnished for Medicare Part B PQRS measures • Breast cancer screening (mammograms) • Colorectal cancer screening • Pneumovax
  • 15. © 2014 IBM Corporation 15 Problem Mechanism for capturing procedures performed at outside institutions is unreliable: • Mammogram • Colo-rectal cancer (CRC) screening • Pneumovax PQRS measures are typically captured using structured Electronic Medical Record (EMR) data
  • 16. © 2014 IBM Corporation 16 Health maintenance in electronic medical record
  • 17. © 2014 IBM Corporation 17 Free-text note in patient’s EMR Not captured in structured data
  • 18. © 2014 IBM Corporation 18 Physician Quality Reporting System (PQRS) measures UNC Healthcare utilized IBM Content Analytics (ICA) to analyze free-text clinical documents to improve the accuracy of our 2012 PQRS measures We used ICA to analyze clinical notes for those patients whose structured EMR data did not indicate compliance with PQRS measure requirements • Mammogram, CRC screening, Pneumovax
  • 19. © 2014 IBM Corporation 19 Results 142 155 236 49 27 247 Mammogram (297) CRC screening (285) Pneumovax (271)NLP Capture of PQRS Measure 0 50 100 150 200 350 300 250 Numberofpatientsoutofcompliance (structureddata)
  • 20. © 2014 IBM Corporation 20 Results 60 53 55 61 Mammogram CRC screening Pneumovax Structured data 48 50 52 54 56 64 62 58 Percentofpatientsmeetingrequirements 60 Structured data + NLP 53 63
  • 21. © 2014 IBM Corporation 21 Follow-up of abnormal test results documented in free-text reports Mammograms | Colonoscopies | Pap tests | Chest x-rays
  • 22. © 2014 IBM Corporation 22 Breast cancer and mammograms • Second deadliest cancer in women • Screening mammograms can reduce mortality 28%of women requiring short-term follow-up for abnormal mammography results do not receive recommended care
  • 23. © 2014 IBM Corporation 23 Mammography reports
  • 24. © 2014 IBM Corporation 24 Performance of NLP for accurately identifying mammography results Characteristics Value Precision 98% Recall 100% F Measure 99% • Precision = positive predictive value If NLP reports a result, the probability that the result is correct • Recall = sensitivity if there is a result in the mammography report, the probability that NLP identifies and accurately classifies the result • F-measure = summary measure of NLP performance Revised NLP model has 100% performance for all characteristics
  • 25. © 2014 IBM Corporation 25 Structured Data is Not Enough  Unstructured data significantly increases the richness and accuracy of analysis and decision making … including paper / faxes Today’s Care Guidelines Only Get You So Far  Not granular enough to deliver on the promise of personalized medicine with data driven insights 1,2 Manual Processes and Traditional Workflow Approaches Don’t Work  Process complexity increases with disease complexity … changing conditions require process adaptability 3 © 2012 IBM Corporation Prediction Results of Knowledge-driven Features plus Data-driven Features !AUC significantly improves as complementary data driven risk factors are added into existing knowledge based risk factors. !A significant AUC increase occurs when we add first 50 data driven features ! " # # $! %&# $ ! %# # $ ! &# $ ' (($) * + , (- . / - $ 0- ' 12 3- 4$! . 5' 6 - 1- 4$ ! 7 8 9 - 31- * 45+ * $ : ; < $ # =&$ # =&&$ # => $ # => &$ # =? $ # =? &$ # =@$ # $ %# # $ " # # $ A # # $ B # # $ &# # $ > # # $ !"#$ % &' ( ) *$+ , $, ) - . &*) / $ Knowledge and Guidelines Data Driven Insights 1. Dijun Luo, Fie Wang, Jimeng Sun, Marianthi Markatou, Jianying Hu,Shahram Ebadollahi, SOR: ScalableOrthogonal Regression for Low- Redundancy Feature Selection and its Healthcare Applications. SDM’12 2. Jimeng Sun, Jianying Hu, Dijun Luo, Marianthi Markatou, Fei Wang, Shahram Edabollahi, Steven E. Steinhubl, Zahra Daar, Walter F. Stewart. Combining Knowledge and Data Driven Insights for Identifying Risk Factors using Electronic Health Records. Under submission at AMIA’12 3. Blind Surgeon Metaphor Problem - W.M.P. van der Aalst, M. Weske, and D. Grünbauer. Case Handling: A New Paradigm for Business Process Support. Data and Knowledge Engineering, 53(2):129-162, 2005 What Have We Learned So Far?
  • 26. © 2014 IBM Corporation 26 A Smarter approach to delivering better outcomes • Build longitudinal “data driven” evidence based population insights • Uncover hidden intervention opportunities • Proactively deliver accountable and personalized care in a patient centered model • Collaborate across caregivers to focus on high cost, high need patients • Prevent at-risk patients from progressing to high cost, high need IBM Strategy to Support Care Coordination:
  • 27. © 2014 IBM Corporation 27 The path forward IBM Smarter Care uncovers valuable insights into lifestyle choices, social determinants, and clinical factors enabling holistic and individualized care to optimize outcomes and lower costs Social determinants such as where one is born, grows, lives, works and ages have direct impact on an individual’s overall health and well being Lifestyle choices have direct impact on an individual’s mental and physical wellness Clinical factors such as specific medical symptoms, history, medications, diagnoses, etc are indicators of an individual’s health
  • 28. © 2014 IBM Corporation 28 © 2013 IBM Corporation 28 Find out more about IBM Smarter Care http://www.ibm.com/smarterplanet/us/en/smarter_care/overview/ Visit my blog or follow me on Twitter http://craigrhinehart.com @CraigRhinehart Craig Rhinehart Director, IBM Smarter Care Strategy and Market Development craigrhinehart@us.ibm.com Thank You

Editor's Notes

  1. Need to determine how this was done. Were the notes flagged by NLP and then verified manually. If so, what was the positive-predictive value?
  2. Find out thresholds.
  3. Validated on 500 mammography reports
  4. Key message: The path forward is IBM Smarter Care… which expresses the power available today to uncover the KEY insights about an individual– their lifestyle choices, social determinants, and clinical factors. And then bring to bear all of that valuable information -- which is distilled, synthesized, and can be acted upon – as never before! Information about lifestyle, such as… do you choose to smoke? Do you choose to exercise? Do you make healthy nutrition a focus? Social determinants…such as where an individual is born, where they live and work, and their age, which can all have a direct impact on overall health and wellness And of course, clinical factors – the more traditional component, such as medical history and symptoms, predisposition to disease, medications, etc This is the recognition that everything matters when it comes to the individual. It’s not just clinical – it is also about their lifestyle choices, and social determinants – that impact overall health and wellbeing, and ability to contribute to community vitality.
  5. Key Points to Make There may be other steps as well. This is a summary slide, just follow the slide.