The Right Way to Build Predictive Models for the Most Vulnerable Patient Populations

The Right Way to Build
Predictive Models for the Most
Vulnerable Patient Populations

© 2021 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
This article is based on a 2020 Healthcare Analytics Summit (HAS 20 Virtual)
presentation titled “Machine Learning, Social Determinants, and Data Selection
for Population Health” by Terri H Steinberg, MD, MBA, FACP, Chief Health
Information Officer, Vice President for Population Health Informatics,
ChrisitanaCare, and Jason Jones, PhD, Chief Data Scientist, Health Catalyst.
The Right Way to Build Predictive Models
Terri H. Steinberg, MD, MBA, FACP
Chief Health Information Officer,
VP for Population Health Informatics,
ChrisitanaCare
Jason Jones, PhD
Chief Data Scientist,
Health Catalyst

The use of artificial intelligence (AI) and,
specifically, predictive models to identify the
most vulnerable patient populations is a
strategic approach to managing population
health initiatives.
By sorting patients based on risk level and
identifying clusters of need, health system
team members can perform outreach and
interventions to maximize the quality of patient
care and the predictive model’s effectiveness.

Building a successful model requires the
right technology stack, human oversight
and intervention, and most importantly,
quality data to fuel the machine.
Disparities in data collection, the type of
data sources available, and limited
interoperability between systems can make
the data component the most challenging
and time-consuming piece of the AI puzzle.

Putting the upfront time and resources into
managing and understanding available data
allows organizations to more easily
build predictive AI models to support
their population health efforts.

Determining Data Sources for Maximum Predictability
Organizations have access to hundreds, if
not thousands, of data points from various
sources—EMRs, Health Information
Exchanges (HIEs), claims, social
determinants of health, etc.—and these
sources often have overlapping data.
Thoroughly understanding the desired model
and outcome allows team members to zero
in on the best data sources to utilize.

It’s important to note that more is not
always better, as overwhelming the model
with unnecessary data points can lead to
confusion and difficulty in maintenance.
The goal should be to build the simplest
model possible with the maximum
predictive power.

Building the model itself is fairly simple when
compared to the data management component.
Therefore, data scientists should avoid forcing a
model built and used for one organization to
work elsewhere.
Patient populations, data collection practices,
data sources and more can vary widely from one
health system to another, rendering a “recycled”
model useless in a new environment.
Data users at each organization must determine
their type of data, source, and desired outcome.

Technology Needs for Predictive Models
Both the input and output of data for a predictive
AI model require certain technology.
In addition to platforms used internally by health
systems, integrated patient tools, such as
member portals and biometric devices, can
provide meaningful data to help segment
populations that may be at risk for poor outcomes.
Population health EMRs optimized to provide AI-
generated recommendations directly into existing
workflows create a seamless experience and
allow team members to provide more timely care.

On the backend, a data and analytics platform
that can aggregate data from all sources is
another must-have in the technology stack.
A robust data platform can perform business
logic and make it actionable, provide visually
appealing dashboards to understand financial
and clinical performance, and perform important
predictions for population health management.

Lastly, an interoperability platform that
allows information to flow freely between
systems is essential.
For example, an open platform like the
Health Catalyst Data Operating System
(DOSÔ) enables health systems to
extract data from various source systems
and aggregate disparate data sets using
healthcare-specific terminology.
The result is powerful analytics and
insights that support predictive models.

Human Oversight for Informed Interventions
While machines are great at recognizing patterns
and running calculations, they don’t take the
place of reasoning, logic, and interventions that
team members must perform.
Human oversight and input are critical in
determining data sets and resources available
and defining the appropriate outreach or
intervention for each outcome or patient
population.
The machine’s output doesn’t necessarily
determine the next steps—it merely allows for
more informed risk stratification and identifies
opportunities for patient engagement.

The Perfect Predictive Model for Population Health
Surprisingly, a successful predictive
AI model will appear to degrade
over time.
A newly launched model may
identify large patient populations as
being at-risk.
But, over time, when paired with an
effective intervention, that model
will appear incorrect.

For example, a model may say that a
patient is at risk for hospital readmission.
In response, the care team puts the
appropriate intervention in place and
prevents the readmission.
Suddenly, it looks as if the model is
wrong because the patient wasn’t
actually readmitted.
When this disconnect occurs, predictive
model users need to capture what the
interventions are and understand why
performance is going down.

Machines can help separate whether the
low performance results from poor
predictability due to data or risk level
changing, or whether it is due to a correctly
implemented intervention.
If the latter is true, then the team has
created a successful model.

The Right Combination of Data,
Technology, and Skillset
Building successful predictive models for
population health requires the right
combination of data, technology, and
human intervention.
The journey requires continual learning,
understanding the data fueling the
outcomes, and optimizing models and
interventions for the most predictive
performance and best quality of care.

For more information:
“This book is a fantastic piece of work”
– Robert Lindeman MD, FAAP, Chief Physician Quality Officer

More about this topic
Link to original article for a more in-depth discussion.
The Right Way to Build Predictive Models for the Most Vulnerable Patient Populations
Artificial Intelligence and Machine Learning in Healthcare: Four Real-World Improvements
Health Catalyst Editors
Three Keys to Improving Hospital Patient Flow with Machine Learning
AI in Healthcare: Finding the Right Answers Faster
Machine Learning Tools Unlock the Most Critical Insights from Unstructured Health Data
Safeguarding the Ethics of AI in Healthcare: Three Best Practices

Terri Steinberg, M.D., M.B.A., FACP is the Chief Health Information Officer and Vice
President of Population Health Informatics at Christiana Care Health System, a large
multi-entity healthcare organization in Delaware. Dr. Steinberg has lectured and
consulted extensively on methods to ensure successful technology adoption by
physicians and nurses, on the positive impact of technology on safe medication practice,
and on the use of technology to drive Population Health management. As a clinician as
well as a software developer, Dr. Steinberg has used her experience to guide the optimal
implementation of clinical systems in a manner that is well-accepted by doctors and nurses. Dr.
Steinberg currently devotes much of her time to population health IT, to provide innovative technology
services to Christiana Care Health System’s care management company, CareLink Care Now.
Other Clinical Quality Improvement Resources
Click to read additional information at www.healthcatalyst.com
Terri H. Steinberg, MD, MBA, FACP

Jason Jones currently serves as Chief Data Scientist at Health Catalyst. Previously, he
served at Kaiser Permanente (KP) in various roles including Research Scientist and VP,
Information Support for Care Transformation. Prior to KP, Jones was a Medical
Informaticist for Intermountain Healthcare. Other roles have included analytic and
marketing leadership positions at Bayer HealthCare, data and information product
development at UnitedHealth Group, and various academic adjunct faculty positions.
Jones received his PhD in Biostatistics from the University of Southern California in
2001. His mission is to leverage data to achieve the Quadruple Aim.
Jason Jones, PhD

Health Catalyst is a mission-driven data warehousing, analytics and outcomes-improvement
company that helps healthcare organizations of all sizes improve clinical, financial, and operational
outcomes needed to improve population health and accountable care. Our proven enterprise data
warehouse (EDW) and analytics platform helps improve quality, add efficiency and lower costs in
support of more than 65 million patients for organizations ranging from the largest US health system
to forward-thinking physician practices.
Health Catalyst was recently named as the leader in the enterprise healthcare BI market in
improvement by KLAS, and has received numerous best-place-to work awards including Modern
Healthcare in 2013, 2014, and 2015, as well as other recognitions such as “Best Place to work for
Millenials, and a “Best Perks for Women.”

The Right Way to Build Predictive Models for the Most Vulnerable Patient Populations

Recommended

Recommended

More Related Content

More from Health Catalyst

More from Health Catalyst (20)

Recently uploaded

Recently uploaded (20)

The Right Way to Build Predictive Models for the Most Vulnerable Patient Populations