CVS Health delivers millions of offers to over 80 million customers and patients on a daily basis to improve the customer experience and put patients on a path to better health. In 2018, CVS Health embarked on a journey to personalize the customer and patient experience through machine learning on a Microsoft Azure Databricks platform.

1. Bringing Personalization to
Customers and Patients at
CVS Health
Michelle Un
Raghu Nakka

2. Agenda
Personalization at CVS
Growth & Challenges
Looking to the Future

3. CVS Health at a Glance
9,990+
retail locations
62M+
ExtraCare card
members
2.5B
prescriptions
managed or filled
4.5M
customers served by CVS
Pharmacy stores daily
Source: https://cvshealth.com/about/facts-and-company-information

4. Understanding CVS customer behavior
Unique Customers
Unique Data
Unique Situations
• Different types of customers means large number of micro-segments which poses
challenges to understand overall behavior of customer
• Pharmacy patient health needs are varied and sometimes unpredictable
• Not a typical grocery store customer
• Hard to predict customer behavior of a convenience shopper
• Data sparsity and dimensionality leads into overfitting issues
• Need to be able to respond to evolving retail and healthcare industries
• Limited ways to influence or incentivize patient health behavior within the pharmacy setting
• Situations like COVID-19 will instantly make static machine learning models either outdated or
invalid
Strict Data Use and Confidentiality Agreement
o Due to patient confidentiality concerns and HIPAA restrictions, Front Store and Pharmacy data and Personalization efforts are kept completely separate
o All use cases are reviewed by legal team to ensure conformance with laws and regulations
o The team does share skills / technical expertise across separate environment to accelerate learning

5. Personalization at CVS

6. Personalization at CVS
Personalization is delivering the right individual experience in the right channel at the right time
The Right Experience...
In the Right Channel...
At the Right Time...
Tailored for
Each Individual Customers
Product
Sales &
Rewards
Service
Loyalty &
Engagement
Mobile Web In StoreReceipt
Call
Center
Location Time Weather Schedule

7. Technology to support Personalization
To move quickly, we leveraged existing technology where suitable and used the cloud to enable an agile
analytics capability
Structured Data
Sources
Files from Data
Sources
Data
Warehouses
Batch Mode
Existing Sources Data Ingestion
Data Layer Features Opportunities Model
Offer
Delivery
Processing Layers
Control-M
Industrialized ‘Next-Best-Action’ Pipeline
DevOps Orchestration Analytics and Compute
DAG
Measurements and
Insights
Microsoft Azure
Test and Learning Experimentation

8. The Personalization Journey
From idea to in-market in less than 6 months, but with constant need to pivot to meet future needs
Built pilot in on-prem
Hadoop environment
20192018
Scaled personalized
offers to 100%
Migrated from on-prem
to Azure Databricks to
enable scale-up
2020
Launched first
personalized offers to
1% customers
Reassess technology for
future growth and use
cases
Scaled test and
learn experimentation
framework
Introduced test and learn
experimentation and
doubled the number of
personalized products
Personalization journey
begins

9. The Personalization Journey
From idea to in-market in less than 6 months, but with constant need to pivot to meet future needs
Built pilot in on-prem
Hadoop environment
20192018
Scaled personalized
offers to 100%
Migrated from on-prem
to Azure Databricks to
enable scale-up
2020
Launched first
personalized offers to
1% customers
Reassess technology for
future growth and use
cases
Scaled test and
learn experimentation
framework
Introduced test and learn
experimentation and
doubled the number of
personalized products
Personalization journey
begins
Team Size

10. Impact of Personalization
There are many use cases enabled through Personalization, which help to enhance customer
experience, improve health outcomes, and drive value for CVS Health
Example Use Case Desired ImpactExample Solution
Determine the best clinical product
to offer a patient given unique
health needs
• Increase prescription refill rate
• Reduce patient gaps in therapy
• Improve condition-level adherence
Determine the best channel to send a
clinical product to a patient
Predict patient's propensity to accept
an offer given outreach based off of
patient behavior
• Increase responsiveness to channel
• Increase adoption of clinical products
• Spend pharmacy time on only most critical
clinical outreaches
Test random assignment of channel, collect
data, then optimize channel assignment using
experimentation or modeling approach
Present most relevant and efficient
coupon offers to ExtraCare members
Create microsegments based off of customer
behavior to provide most relevant offers
• Increase engagement of active customers
• Re-engage with lapsed customers
Improve interpretation of patient's
responses
Analyze past responses and improve
interpretation through rule-based engine;
explore and leverage natural language
processing for real-time interpretation
• Increase responsiveness to channel
• Increase adoption of clinical products
• Improve patient experience

11. Example Use Case:
Pharmacy
Problem
• One barrier to medication adherence that many
patients face is picking up medication on a
regular basis, due to a variety of barriers, such as
forgetfulness, cost, access, side effects, etc.
Example Solution
• Understand medication adherence profile and
barriers to medication adherence
• Use machine learning to determine what
products and services may help a patient based
on their specific needs (i.e. automated refills,
reminders and notifications)
• Use test and learning experimentation to
determine the best channel, timing, and
messaging to promote and package the service
to the patient
Offer automatic refill
service to patient
Offer the service to the
patient through a SMS
Emphasize the convenience
offered by the service
in the messaging

12. Example Use Case:
Retail Front Store
CouponOffer at
Register
Coupon influences the
next store trip
Presence of offer
expands basket size
during the trip
Problem
• Growing the engagement of most
valuable customers. Re-engage lapsed
customers and engage all active product
shoppers
Example Solution
• Personalize communication, provide
relevant and exciting offers using data from
customer profile

13. Results driven through Personalization
Data and insights driven strategy in retail pharmacy help to improve to medication adherence
Pharmacy
RESULTS
1.6%
overall
improvement
in adherence
PREDICTION
Optimized Offers
Refill rate reminder
timing
Preferred
messaging/
effectiveness
Prescriber outreach
effectiveness
KEY PROFILE ELEMENTS
Adherence
likelihood
Medication
profile
Cost profile and
cost sensitivity
Offer and
response history
Prescriber relationship
Source: Enterprise Analytics, 2019; Image Source: CVS Health Creative Resource Library, accessed 2019

14. Growth & Challenges

15. Enabling Speed to Market
Cloud-based environment enabled us to get to market quickly, expand the platform with the growth of
the team, and rapidly introduce new improvements to the application
Ease of Use Collaboration
• More flexibility to spin up and down
different types of clusters to meet
emerging project needs and
changing team size
• Not restricted to physical hardware
constraints
• Less focus on Spark tuning due to
performance and autoscaling
features
• Databricks workspace centralizes
assets to make development easier,
including interactive notebook, cluster
management and performance
monitoring, metastore
• Less worrying about infrastructure
support and more focus on analytics
• Easy to integrate new Azure services
or supported services into our
environment
• Easy to onboard new users and
teams to integrated workspace
environment through Databricks
• Easy to collaborate using Databricks
notebooks
3Ease of Use2Scalability1

16. Growth & Performance Challenges
With quick growth, we encountered many challenges along the way, many of which we continue to
tackle
Handling
Big Data
Cost
Management
Evolving
Technology
• Size of data causes breakdown of production
pipeline while adding new features
• Growing team has varying exposure to Spark so use
cases outpace optimization
• Retail and healthcare industry not evolving as fast as
cloud technology industry
• New solutions need to be developed within the
constraints of the retail and healthcare industry (i.e.
security, compliance, archival regulations)
• Different clusters for ETL, model training, and
feature creation
• Code optimization and sampled data for
more efficient cluster usage
• Utilize features like cluster policies, pools, etc.
Finding Talent
• Difficult to find cross-functional skillsets with data
science, engineering, and DevOps background
• New / changing technology means limited pool of
talent
• Hiring talent with Python helped bridge the gap
between data science and engineering
• Increase training and opportunities to explore
new technology to grow the skillset
• Increasing size of team leads to high cost
• Cost visibility can be fragmented and difficult to
monitor cost
• Some optimization from Delta, but comes with
challenges
• Focus on partition sizing and cluster usage as
core concepts
• Assess alternate options that meet specific
requirements
• Advocate for features to be added where
needed
Solutions & Areas for ExplorationChallenge

17. Feature
Engineering
Machine Learning Journey
CVS is making strides towards leveraging machine learning for Personalization but with this comes a
unique set of challenges
Model
Training
Implement
Solution
• Automate model training pipeline, making it
generalizable across many use cases
• Modernize architecture that expands algorithm
options
Collaboration
• Blended roles and responsibilities
• Hand-off between data science and data
engineering can be clunky
• Increase cross-training and encourage ML
Engineer skillset
• Integrated development and deployment teams
in an Agile environment
• Micro-segmentation and disparate use cases
require many models to be trained and
implemented in parallel
• Robust model selection process
• Create centralized features which can be
leveraged across many use cases
• Explore deep learning and reinforcement
learning
• Exploring optimization software like
IBM CPLEX
• Explore tools like Kubeflow and MLFlow to
support model management and tracking
Solutions & Areas for ExplorationChallenge
• Complex feature engineering requires a lot
of compute power
• Optimization Layer constraints outside of
model increases complexity
• Model deployment is a manual process
• Limitations to each modeling approach

18. Looking to the Future

19. Finding the Right Tool for the Job
As future use cases evolve, there is a need to continually explore and new tools and technology to
existing tech stack
Real-Time Use CasesMachine LearningStorage and Compute Orchestration
Azure Cosmos DB
Azure Event Hubs

20. Questions?

21. Feedback
Your feedback is important to us.
Don’t forget to rate and
review the sessions.