The enterprise data warehouse (EDW) at Intermountain Healthcare went live in 1998. The EDW at Northwestern Medicine went live in 2006. Dale Sanders was the chief architect and strategist for both. The business inspiration behind Health Catalyst was, in essence, to create the commercial availability of the technology, analytics, and data utilization skills associated with these systems at Intermountain and Northwestern. Lee Pierce assumed leadership of the Intermountain EDW in 2008. Andrew Winter assumed leadership of the Northwestern EDW in 2009, and transitioned leadership of the EDW to Shakeeb Akhter in 2016. This webinar is a fireside chat among friends and colleagues as they look back across their healthcare IT decisions to answer these questions: What did we do right and what did we do wrong? What advice do we have for others in this emerging era of Big Data? What does the future of analytics and Big Data look like in healthcare?

1. 20 Years in Healthcare
Analytics & Data Warehousing
What did we learn? What’s the future?
Shakeeb Akhter - Director,
Enterprise Data Warehouse
at Northwestern Medicine
Lee Pierce - Healthcare Chief
Data Officer at Sirius and former
Chief Data Officer at
Intermountain Healthcare
Dale Sanders, President of
Technology at Health Catalyst

2. • People
• Processes
• Technology
• What did we do right?
• What did we do wrong?
• What’s the future?
Many thanks to Lee and Shakeeb
Agenda

3. © 2018
Health
Catalyst
I learned what was right by first doing what was wrong.
Luckily, I watched others do what was wrong and learned from them
without suffering their wounds as mine.
Generally Speaking…
3

4. People
4

5. © 2018
Health
Catalyst
What did we do right?
• Hired a balanced combination of social skills, domain skills (clinical and financial),
and technical skills
• Good balance of centralized/decentralized development of these skills… 60/40 split
What would we have done differently?
• Overlooked the cultural issues of ”data”… threatened legacy source systems’
teams… Intermountain the IDN vs Northwestern the AMC
• Too much dependence on matrixed technology resources, esp DBAs and Sys
Admins who were skilled in transaction databases, not analytics
What are we thinking about the future?
• Data science, non-relational technical skills
• A new role: The Digitician
• Reality of the 8% Folly
People - Dale
5

6. © 2018
Health
Catalyst
Data as a Strategic Corporate Asset
This should be
every healthcare
CEO’s strategic
data acquisition
roadmap
6

7. © 2018
Health
Catalyst
What did we do right?
• Hired amazing data professionals - dedicated, smart, committed - treated them right
• Embraced early support from business and clinical leaders that allowed us to start
early: Dr. Brent James, Dr. Homer Warner
• Partnered with & empowered the data analysts to be primary producers of analytics
• Engaged business leaders in Data and Analytics strategy and execution
What would we have done differently?
• More thoughtful and coordinated growth, hiring of data analysts across the business,
earlier job description standardization and smarter growth
• More bold steps to organize analytics as centrally managed, locally deployed
What are we thinking about the future?
• Improved data literacy for leaders = how to ask for analytics help, how to use
insights to improve decision making and change business processes = value
People - Lee
7

8. © 2018
Health
Catalyst
What did we do right?
• Small, highly skilled team at the start resulted in agility and faster product to market
• Multiple tangentially related functions developed within the EDW team that resulted
in self-sufficiency
• Recruited EMR Application Analysts with exposure to Business Intelligence
What would we have done differently?
• Develop additional roles to accommodate growth and scale with the organization
• Recruit clinical and operational SMEs to assist with understanding of data
What are we thinking about the future?
• Develop specialized roles in order to scale for future growth
• Blur the lines between EDW & Analytics via hybrid resources
People - Shakeeb
8

9. Processes
9

10. © 2018
Health
Catalyst
What did we do right?
• Design and code reviews
• Lightweight data governance – govern to the least extent necessary for the greatest
common good
• Using EHR logs to analyze & understand workflow (the first ‘Meaningful Use’)
• Running the data warehouse like a small business
What would we have done differently?
• Prioritization management, demand vs. capacity
• Managing expectations of data and analytic quality validation
What are we thinking about the future?
• AI/data science changes almost everything... design reviews, data governance,
analytic validation
• Study the DevOps of AI
10
Processes - Dale

11. © 2018
Health
Catalyst
• Enabled by bio-integrated sensors, patients
hold more data about themselves than the
healthcare system
• Their data is constantly being updated and
uploaded to cloud-based AI algorithms
• Those algorithms diagnose the patient’s
condition, calculate a composite health risk
score, and recommend options for
treatment or maintaining health
• The algorithm suggests options for a “best
fit” care provider and the ability to socially
interact with other patients like them
Future of Diagnosis and Treatment
11
• The patient engages with the care provider,
enabled with the output of the AI algorithms

12. © 2018
Health
Catalyst
What did we do right?
• Vision for the use of data – clinical quality improvement
• Focused on real business/clinical use-cases and needs
• Discipline (in many use cases) to implement insights – close the loop
• Practical data governance – project-based, use-case driven, clear business value
What would we have done differently?
• Require analytics return-on-investment planning and measurement for large
analytics projects, document ROI, communicate successes and repeat
• Establish (and enforce) better standard development practices from the beginning
(EDW/ETL specifically), reducing maintenance required after dev.
• Should have been more bold about data governance value and investment
What are we thinking about the future?
• More business discipline to use insights generated with more focus on end value
12
Processes - Lee

13. © 2018
Health
Catalyst
13
Processes - Lee

14. © 2018
Health
Catalyst
What did we do right?
• Single EDW for Research and Operational Analytics
• Small teams leveraged agile principles (informally) to start
• Data Steward Approval + Power User Model
• Report Deployment process
What would we have done differently?
• Avoid waterfall, embrace agile
• Fail faster and more often
• Bring the customer to the table
What are we thinking about the future?
• Dev + Data Ops; Agile Product Teams, source control, automated testing and deployment
• Investing heavily in Data Management; Data Quality and Master Data Management
• Embracing Agile tools & methods
Processes - Shakeeb
14

15. © 2018
Health
Catalyst
The primary goal of Data Ops is to achieve Customer Satisfaction with DataAssets &Analytical
Solutions across the enterprise
• Standardization: Establish standardized tool sets and
processes to increase productivity of data teams
• Cross-Functional Teams: Break-Down silos within data
teams (EDW/Analytics) by establishing cross-functional
teams consisting of Data Architects, Analytics Consultants,
Report Writers, ETL Developers and Data Scientists
• Customer-Focus: Increased alignment with customer
focus and priorities by establishing customer-centric
product teams to serve data needs for operating units and
system functions
• Value-Add: Provide continuous delivery of analytical
insights to enterprise customers in order to establish a
data-driven culture
• Customer Satisfaction: Increase customer satisfaction due
to customer-focused teams and alignment with customer
priorities
Increased
Customer
Satisfaction
Continuous
Delivery of
Analytical Insights
Cross-Functional,
Customer-Centric,
Product Teams
Standardized Tools
and Processes
15
Dev Ops - Goals

16. © 2018
Health
Catalyst
Data Ops Product Teams contain all skills required to transform data from its raw form to an
end-user analytical deliverable; report, dashboard, KPI, or analytical application. Possible roles,
and product teams are displayed as a sample below.
Product Team A
NOTE: The above is a draft representation. Specific Product Teams are being defined byAnalytics and EDW teams and will be formed accordingly.
Data
Architect
Analytics
Manager
Sr. Data
Architect
ETL
Developer
Data
Quality
Analyst
Sr.
Analytics
Associate
Analytics
Associate
Product Team B
Data
Architect
Analytics
Manager
Sr. Data
Architect
ETL
Developer
Data
Quality
Analyst
Sr.
Analytics
Associate
Analytics
Associate
• Analytics Manager is accountable for the
customer satisfaction with EDW
deliverables, including self-service data &
reporting applications
• Sr. DataArchitect is accountable for the
efficient & effective integration of data in
EDW and the necessary data structures to
support reporting
• Analytics Manager & Sr. DataArchitect will
collectively set priorities and timelines and
escalate to leadership as needed
• Staff level roles work collaboratively as one
collective team with the customer’s
satisfaction of EDW tools provided as the
primary outcome metric
• Specific # of resources & coverage of
product teams will vary based on expected
demand & resource availability, but FTEs
will be a part of multiple product teams
16
Dev Ops – Product Teams

17. Technology
17

18. © 2018
Health
Catalyst
What did we do right?
• Ignored the enterprise data model in favor of late binding… didn’t need expensive ETL tools
• Took advantage of tried and true SMP architectures
• Ignored the early love affair of Hadoop
• Blended text with discrete data when nobody else was doing it
• Picked Microsoft when nobody else was doing it
What would we have done differently?
• Too much late binding data modeling
• Jumped into massively parallel processing too soon… let the pure technologists talk me into it
• Too much faith in an “enterprise standard” business intelligence tool
What are we thinking about the future?
• Read-only, batch-oriented relational data warehouses are already outdated
• Hybrid transaction & analytic architectures… Lambda and Kappa architectures
Technology - Dale
18

19. External
Data
Sources
EMR
SQL
HL7
X12
FHIR
Flat-files
XML
Data
Integration
Batch Data
Realtime Data
Mirth,
RabbitMq
Catalyst
Data Engine
SQL
Big Data
.NET
DOS
App Cluster
Apps
Microservices
Highly Available
Horizontally Scalable
Angular, D3, .NET, Java, Docker,
Kubernetes, JSON
Datamart Designers & Tools
SAMD, SMD, Atlas, Ops Console, Analytics Portal
Data & Compute Cluster
SQL Server, Hadoop, Spark, ElasticSearch
Transactional data store
SQL, Shared Disk
DOS Marketplace
Apps, Content, AI models
Catalyst AI
Engine
Catalyst.ai
healthcare.ai
.NET, R, Python
Azure
Azure
AI Cluster
Spark, R, Python
SQL
FTP
HL7
FHIR
SQL
HTTP
FHIR
HL7
External
Apps
EMRs
Reports
The Health Catalyst Data Operating System Architecture

20. © 2018
Health
Catalyst
What did we do right?
• Collaborated with others– HDWA->HDAA, HMA
• Purchased and implemented Tableau
• Resisted ongoing pressure to move data to canonical data models (i.e. Oracle genomics)
• Built an analytics reference architecture to use to rationalize tools/tech decisions
What would we have done differently?
• Would have been more thoughtful about migration from one tool to another – cost to migrate
vs. value realized
• Build more meaningful working relationship between EMR vendor(s) and healthcare analytics
programs, focus on opportunities to deliver value through analytics
What are we thinking about the future?
• Embrace the cloud – simplify administration of technologies, computing power, flexibility
• Personalized analytics – n of 1 – will require new data sets, more external data
Technology - Lee
20

21. © 2018
Health
Catalyst
Technology - Lee
21

22. © 2018
Health
Catalyst
What did we do right?
• Maintained Microsoft BI Stack rather than a proliferation of various BI tools
• Did not go all-in on Hadoop early
• Waited to modernize the platform until market was more mature
What would we have done differently?
• Scale out legacy infrastructure in order to distribute workloads
What are we thinking about the future?
• Transform EDW into an ecosystem of various technology and tools
• Mission Critical Work-Loads on Premise
• Cloud for Advanced Analytics, Dev/Test, Disaster Recovery and ‘Cold’ storage
• Utilize Tabular Models for Self-Service
• High Availability
Technology - Shakeeb
22

23. © 2018
Health
Catalyst
Current state of a data warehouse
23
Traditional Data Warehousing Approach

24. © 2018
Health
Catalyst
Modernized Data Warehouse Architecture
2
4
Cloud Scale & Performance. Single-Query Model.Data Science.
24

25. © 2018
Health
Catalyst
NMEDW Modernized Architecture
2
5
The new architecture separates and scales out workloads to provide improved performance and high
availability while leveraging the cloud for big data stores, advanced analytics workloads, and infinite
scalability
25
Staging / ODS
ODS
Storage 14TB, 256GB RAM
Cores 16
ODS_2
Storage 18.5TB, 256GB RAM
Cores 10
Cerner
Storage 15TB, 256GB RAM
Cores 10
Sensitive Data
Storage 100GB, 32GB RAM
Cores 4
Clarity Test
Storage 16TB*, 32GB RAM
Cores 4
High Availability Active/Passive
Clarity West_A
Storage 16TB*, 384GB RAM
Cores 16
Clarity West_B
Storage 0TB*, 384GB RAM
Cores 16
Extract, Transform, Load (ETL)
SSIS
Storage 512GB, 512GB RAM
Cores 16
Clarity Console
Storage 100GB, 8GB RAM
Cores 4
Clarity Console Test
Storage 100GB, 4GB RAM
Cores 2
Data Integration Data Warehouse Semantic
Caboodle
Storage 6.4TB, 256GB RAM
Cores 12
Caboodle Test
Storage 6.4TB*, 32GB RAM
Cores 4
Applications DB
Storage 3TB, 128GB
Cores 4
Ancillary Environments
Storage 4TB, RAM 128GB
Cores 6
High Availability Active/Active
EDW/IDS
Storage 26TB, 500GB RAM
Cores 16
EDW/IDS
Storage 0TB*, 500GB RAM
Cores 16
SSAS
Storage 6TB, 512GB RAM
Cores 16
SSRS
Storage 256GB, 256GB RAM
Cores 8
Tableau
Storage 1TB, 256GB RAM
Cores 8
BusinessObjects Test
Storage 230GB, 16GB RAM
Cores 2
SlicerDicer
Storage 230GB, 96GB RAM
Cores 6
High Availability Active/Passive
BusinessObjects_B
Storage 0GB*, 16GB RAM
Cores 4
BusinessObjects_A
Storage 230GB, 16GB RAM
Cores 4
SlicerDicer Test
Storage 230GB, 16GB RAM
Cores 4
High Availability Active/Passive
Applications_A
Storage 512GB, 8GB RAM
Cores 1
Applications_B
Storage 0GB*, 8GB RAM
Cores 1
Ancillary
Storage 3TB, 128GB RAM
Cores 6
Azure SQL
DB
HDInsigh
t
Cortana
Analytics Suite
Azure Machine
Learning
Data Factory
Data Catalog
Cognitiv
e
Services
Cortana
Bot
Framework
Data Lake Store
Data Lake
Analytics
Azure Storage
Azure SQL DB
Stream
Analytics Event Hubs
IoT Suite
IoT Hub
Web/
Mobile App

26. © 2018
Health
Catalyst
Healthcare Analytics Summit 18
Sept. 11-13, Salt Lake, Grand America Hotel
TOBY COSGROVE, MD
former CEO and President of
Cleveland Clinic (2004-2017),
who as a cardiac surgeon
performed more than 22,000
operations and holds 30 patents
for medical innovations
KIM GOODSELL
the actualized ‘genomified,’ quantified,
digitalized “patient of the future," her debut at
the 2014 Future of Genomic Medicine
conference made headline news
announcing— “The patient from the future,
here today”
DANIEL KRAFT, MD
a Stanford and Harvard trained physician-
scientist, inventor, entrepreneur, and
innovator, Kraft is the Founder and Chair of
Exponential Medicine, a program that
explores convergent, rapidly developing
technologies and their potential in
biomedicine and healthcare
BRENT JAMES, MD
former Chief Quality Officer at
Intermountain Healthcare - known
internationally for his work in
clinical quality improvement,
patient safety, and the
infrastructure that underlies
successful improvement efforts
PENNY WHEELER, MD
President and Chief Executive
Officer of Allina Health,
returns a second time as one
of the most popular HAS
speakers ever
MARC RANDOLF
Co-founder of Netflix, Marc will
share the Netflixed story: how a
scrappy Silicon Valley startup
brought down Blockbuster and
the lessons that could be
applicable to healthcare
JILL HOGGARD GREEN
PhD, RN, Chief Operating Officer – Mission
Health and President – Mission Hospital,
and recently named to the 2017 Becker’s
Healthcare list of the country’s top Women
Hospital and Health System Leaders to
Know
ROBERT WACHTER, MD
global leader in healthcare safety,
quality, policy, IT; Chair of the
Department of Medicine, University of
California, San Francisco; best-selling
author, “The Digital Doctor: Hope, Hype
and Harm at the Dawn of Medicine’s
Computer Age”
More highlights
4 Digital Innovators (Keynotes)
AI Showcase (10 walkabout case studies)
Digitizing the Patient Showcase (10-12 stations)
28 Educational, Case Study, and Technical Breakouts
24 Analytics Walkabout Projects
More Networking (Introducing “Brain Date”)
CME Accreditation For Clinicians
5-Star Grand America Hotel Experience
96 Total Presentations
National keynotes
Employer
Innovation
Scott
Schreeve
MD, CEO, Crossover Health
Payer
Innovation
Kevin
Sears
Executive Director of Marketing
and Network Services, Cleveland
Clinic
Biosensor
Innovation
John
Rogers
PhD, Founding Director, Center
Bio-Integrated Electronics,
Northwestern University
Pricing
Innovation
Gene
Thompson
Project Director, Health City
Cayman Islands

27. Thank You!