AI in Healthcare: Finding the Right Answers Faster

AI in Healthcare:
Finding the Right
Answers Faster

© 2020 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
Jason Jones, PhD
Chief Data Scientist
This report is based on a 2019 Healthcare Analytics Summit
presentation given by Jason Jones, Chief Data Scientist Officer,
Health Catalyst, entitled, “Getting to the Wrong Answer Faster:
Shifting to a Better Use of AI in Healthcare.”
AI in Healthcare

AI in Healthcare
Data and analytics can be the driving
force behind the successes or failures
of a health system.
To transform healthcare delivery,
data is critical—but only if the data
leads you to the right conclusion.
Wrong conclusions within your
analytics can cause suboptimal
outcomes for patients and wasted
attempts to utilize artificial
intelligence (AI) in healthcare.

AI in Healthcare
Commonly used analytic methods—
particularly with AI in healthcare—can
often lead analysts and leaders to
unknowingly draw the wrong
conclusions.
Therefore, it is imperative that data
leaders understand and leverage
important strategies and tools to
derive the right conclusions and
recognize the wrong answers.

Data Stewardship—Key to Data Leadership
Leaders in healthcare have a significant
amount of power with the use of AI/ML; yet,
with the same data sets, different leaders
can draw completely different conclusions.
Therefore, data leaders and analysts have
a responsibility to act as stewards of data
and help colleagues and team members
use data correctly so they can arrive at
the right answers faster.

Data Stewardship—Key to Data Leadership
It is not uncommon for data users to
arrive at the wrong answers, but have
no idea, because the answers still look
aesthetically pleasing.
That’s why data stewards play a key
role in overseeing appropriate use,
and display, of data.

Laying the Right Foundation for AI in Healthcare
Before using any form of AI in healthcare,
the key is to first identify the problem that
needs to be solved.
It is recommended that AI/ML projects
have at least two years’ worth of historical
data, a statistical process control chart
that clearly identifies the population, and
a clearly defined outcome.

Laying the Right Foundation for AI in Healthcare
One of the first mistakes organizations make
with data and AI/ML is leveraging AI/ML without
the proper foundational data, which inevitably
leads to the wrong conclusions based on
insufficient data and wasted resources that
can take years to recover.
When organizations have laid a strong data
foundation, AI/ML take raw, tabular data
(Figure 1) and turn it into something people
can use to make decisions (Figure 3).
Figure 1. A tabular list of cancer
diagnosis rates by region.

The Right Data Visualization Tools Drastically
Change Data Interpretation
Another step in the journey to find the right
answer is to understand which data
visualization tool is right for your data sets.
The right data visualization tool will radically
change the way people consume, see, and
then interpret data.
For example, if someone was interested in
buying a house, but heard there were higher
rates of cancer in certain areas of the region,
that person might want to view the cancer rates
vs. location data in order to identify which
areas have higher rates of cancer (where to
avoid buying a house).

The data could be displayed myriad ways.
In the Figure 1 table shown previously, the
cancer by location data is displayed in a
tabular list that shows each location where
cancer has been reported, grouped into
geographic regions.
For example, there are four reported
cancers in region A01, two cancers in
region A02, etc.

Another way to display the same data is to
use a scatterplot (Figure 2), a powerful data
visualization tool that allows users to more
easily consume and interpret data.
Figure 2. A scatterplot showing
cancer diagnosis rates by region.

A person can clearly identify cancer-free
regions, empowering data-driven decisions.
The right visualization tool also allows
users to take the data one step further
(Figure 3) and identify trends, patterns,
and clusters in the data to target
opportunities for improvement.
Figure 3. A scatterplot showing cancer diagnosis
rates by region showing cancer free areas.

However, were the person to use AI/ML, in
addition to using data to make a decision,
she may find that there is an even better
way to display the exact same data.
A bar chart (Figure 4) overlaid with an
algorithm that predicts cancer rates.
The blue line represents the bell curve
based on the actual data (cancer rates
based on geography) and the purple line
(AI/ML) is what was expected if there was
no relationship between cancer diagnoses
and geography.
Figure 4. A barchart showing cancer
diagnosis rates by region.

Because the bell curve of the actual data
aligns so closely with the AI/ML bell curve,
there is strong evidence that there is no
relationship between cancer diagnoses
and region.
Therefore, someone looking to buy a
house in this region should not base
their decision on cancer rates by region.

Same Data, Different Conclusions:
Which Conclusion Is Correct?
The cancer rates by region example illustrates that the data never changed,
only the way the data was displayed.
It is crucial for data leaders to understand
how data visualization tools can drive
people (or health organizations) to
the right answer or the wrong answer.

For example, the East Africa Institute of
Certified Studies’s (ICS) attempted to
improve academic performance of
Kenyan kids in grade school.
At first, ICS provided more books
(rather than the one book for the entire
classroom), flip charts, and teachers.
The changes made no difference. In
fact, ICS saw an increase in inequity.

The initial hypothesis was that the kids
who benefitted from the books were
already high performers, and with their
own book and more individual attention
from teachers, the high performers will
keep outperforming the low performers,
increasing the disparity.

A leader at ICS mentioned the findings to a
colleague at the World Health Organization
(WHO), who suggested school absences
due to worm-based illnesses might be the
problem.
It turned out that many of the students
were missing a significant amount of
classroom time due to worm infections.

ICS decided to implement deworming
days at school, an opportunity for
students to safely seek treatment for
their illnesses at school.
Overtime, ICS saw school absenteeism
decrease by 25 percent and income
levels increase by 20 percent levels
over 10 years.

Although ICS’s first attempts at
academic improvement led it to the
wrong answers, the leadership
collaborated with leaders at WHO and
were willing to try something new in an
attempt to get to the right answer.
Because of its collaborative efforts,
humility, and commitment to improve
academic performance, ICS identified
the right answer—deworming programs
at schools—that caused massive
change for Kenyan kids, both now and
in the future.

Look Ahead:
Five Common Roadblocks for AI in Healthcare
Although AI in healthcare seems ubiquitous, and even straightforward, there
are common challenges that arise. Five occur as data analysts and leaders
try to leverage AI/ML to get to the right answers :
Predictive
Analysis
Before
Diagnostic
Analysis
Leads to
Correlation
but Not
Causation.
Change
Management
Isn’t
Considered
Part of the
Process.
The Wrong
Terms to
Describe the
Work.
Trying to
Compensate
for Low Data
Literacy
Resulting in
Unclear
Conclusions.
Lack of
Agreement
on
Definitions
Causes
Confusion.
1 2 3 4 5

Look Ahead:
1. Predictive Analysis Before Diagnostic Analysis Leads to Correlation―Not Causation
In the Gartner Analytic Ascendency Model
(Figure 5), the diagnostic analysis does not
always have to precede predictive analysis.
Predictive analysis, an exercise of correlation
that does not reveal the why behind the
correlation, is sometimes easier to focus
on/identify before someone focuses on the
diagnostic analysis, trying to draw causation
and understand why something happens.

Look Ahead:
1. Predictive Analysis Before Diagnostic Analysis Leads to Correlation―Not Causation
When trying to understand data,
sometimes analysts become too
focused on following the Ascendancy
Model with exactness—trying to
understand the ‘why’ (Diagnostic
Analytics step) and then the ‘what’
(Predictive Analytics step).
There are instances in which
identifying the ‘what’ will then help
someone identify the ‘why’, but it
requires a flexible mindset.
Figure 5. The Gartner Analytic Ascendancy Model

Look Ahead:
2. Change Management Isn’t Considered Part of the Process
In the Gartner Model above, the “prescriptive
analytics” benchmark seems like a technical
challenge; it is a challenge with leadership.
At this point in an organization’s analytics
journey, the changes that leadership need to
make are obvious, but it takes unwavering
leaders to implement new processes that
effect real change.
Without change management, the insight
gained from analytics reach a dead end and
the work to identify opportunities for
improvement were in vain.

Look Ahead:
3. The Wrong Terms to Describe the Work
Rather than using the term “We will evaluate
your program” to measure a program’s
success, data architects, analysts, and
leaders should use the term “Let’s work
together to optimize your program”.
This engages team members and emphasizes
the collaboration aspect, resulting in more
effective, data-informed programs.

Look Ahead:
3. The Wrong Terms to Describe the Work
Using words like “work together” and “optimize”
(instead of “We will evaluate…”) are more
welcoming to team members and send a
message that the analytics team is there to work
with them, rather than judge/evaluate their work.

Look Ahead:
4. Trying to Compensate for Low Data Literacy Resulting in Unclear Conclusions
To overcome decision makers’ lower levels of data literacy,
data architects often oversimplify information.
Instead they should leverage the features
of AI/ML in “standard” reporting—such
as adding confidence limits, computer
forecasting, etc.—to facilitate
interpretation and make
conclusions clear and
easier to understand
for leaders.

Look Ahead:
4. Trying to Compensate for Low Data Literacy Resulting in Unclear Conclusions
Including more information, rather than
less, provides more context, decreases
guessing, and empowers leaders to make
decisions based on a complete picture.

Look Ahead:
5. Lack of Agreement on Definitions Causes Confusion
The idea of a single version of truth is
illusory and does not exist in healthcare.
Leaders need to pursue convergence of
evidence and discuss all possible options,
then make an informed decision centered
on that evidence-based discussion.
If teams focus on agreeing on one single
idea of truth, they will never progress past
that point because it does not exist.

Look Ahead:
5. Lack of Agreement on Definitions Causes Confusion
That is why it is imperative to leverage the
diversity of thinking that comes from
multidisciplinary teams—to brainstorm
ideas, define the problem based on
everyone’s input, and then implement
changes to address that problem.

AI in Healthcare Isn’t Enough Without Humans
AI/ML bring power, utility, and efficiency to the
healthcare world, but it does not replace the
invaluable role that humans play.
Analytic processes require guidance from data
leaders and stewards in order to draw the right
conclusions.

AI in Healthcare Isn’t Enough Without Humans
Specific AI/ML tools and techniques are both
useful and attainable, but they are not enough
for healthcare organizations to arrive at the
right answers.
In order to eliminate the wrong answers faster
and ultimately find the right answer, health
systems need a collaborative approach, an
understanding of data and analytic
processes, and leaders who remove common
barriers and stay focused on moving forward.

For more information:
“This book is a fantastic piece of work”
– Robert Lindeman MD, FAAP, Chief Physician Quality Officer

More about this topic
Link to original article for a more in-depth discussion.
AI in Healthcare: Finding the Right Answers Faster
Meaningful Machine Learning Visualizations for Clinical Users: A Framework
Valere Lemon, MBA, RN, Senior Subject Matter Expert; Alejo Jumat, User Experience Designer, Sr.
Artificial Intelligence in Healthcare: A Change Management Problem
Health Catalyst Editors
Machine Learning Tools Unlock the Most Critical Insights from Unstructured Health Data
How Artificial Intelligence Can Overcome Healthcare Data Security Challenges and Improve Patient Trust
Healthcare Data Management: Three Principles of Using Data to Its Full Potential
Sean Whitaker

Other Clinical Quality Improvement Resources
Click to read additional information at www.healthcatalyst.com
Jason Jones currently serves as Chief Data Scientist at Health Catalyst. Previously, he served at
Kaiser Permanente (KP) in various roles including Research Scientist and VP, Information Support
for Care Transformation. Prior to KP, Jones was a Medical Informaticist for Intermountain
Healthcare. Other roles have included analytic and marketing leadership positions at Bayer
HealthCare, data and information product development at UnitedHealth Group, and various
academic adjunct faculty positions. Jones received his PhD in Biostatistics from the University of
Southern California in 2001. His mission is to leverage data to achieve the Quadruple Aim.
Jason Jones, PhD

Other Clinical Quality Improvement Resources
Click to read additional information at www.healthcatalyst.com
Health Catalyst is a mission-driven data warehousing, analytics and outcomes-improvement company
that helps healthcare organizations of all sizes improve clinical, financial, and operational outcomes
needed to improve population health and accountable care. Our proven enterprise data warehouse
(EDW) and analytics platform helps improve quality, add efficiency and lower costs in support of more
than 65 million patients for organizations ranging from the largest US health system to forward-thinking
physician practices.
Health Catalyst was recently named as the leader in the enterprise healthcare BI market in
improvement by KLAS, and has received numerous best-place-to work awards including Modern
Healthcare in 2013, 2014, and 2015, as well as other recognitions such as “Best Place to work for
Millenials, and a “Best Perks for Women.”

AI in Healthcare: Finding the Right Answers Faster

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to AI in Healthcare: Finding the Right Answers Faster

Similar to AI in Healthcare: Finding the Right Answers Faster (20)

More from Health Catalyst

More from Health Catalyst (20)

Recently uploaded

Recently uploaded (20)

AI in Healthcare: Finding the Right Answers Faster