Document owner: Dale Sanders
Date: March 2013
Guidance for Evaluating and
Choosing an Analytics Solution in
General criteria for the options assessment
Framing the analytic options assessment
What are the factors affecting analytics in the industry?
What are the guiding concepts and philosophies?
What’s the trajectory of the industry and how should we
Specific criteria for choosing an analytic solution
Technical and cultural change management
Vendors in the space… and its crowded
General Criteria For Options
Completeness of Vision
Lessons from the past, understanding of the present, vision of the future
Ability to Execute
References and scalability
Time to Value
Culture and Values of Senior Leadership
Do they align with yours?
Technology Adaptability & Supportability
How fast can the system adapt to the market and your unique needs for differentiation?
Total Cost of Ownership
Will they be around in 8 years? If not, can you live without them?
I score these on a 1-10 basis, for each vendor and option
Framing the Analytic
The Core Analytic Issue
Healthcare Value =
Quality of Health
Cost of Care
Everything we do analytically should relate
back to a better understanding of both the
numerator and denominator, in an
integrated fashion. They are inseparable.
Technology x Change = Solution
“The prerequisite is the technological infrastructure. The harder
thing is to get the set of skills…and that includes not just the
analytical skills, but also a set of attitudes and understanding of
the business. And then the third thing which is the subtlest, but
perhaps the most important is this cultural change…this
attitude about how to use data. There are a lot of companies
who think they are using data…but historically that sort of data
has been used to confirm and support decisions that had
already been made by management, rather than learn new
things and discover what the right answer is. So the cultural
change is for managers to be willing to say, „That‟s an
interesting problem, that‟s an interesting question. Let‟s set up
an analysis to understand it; let‟s set up an experiment.” They
have to be willing to open up and in some ways show some
vulnerability and say “Look we are open to the data.”
Erik Brynjolfsson, the Schussel Family Professor of
Management Science at the Massachusetts Institute of
Technology, Director of the MIT Center for Digital Business
Technology Adaptability: The
Evolving Data Ecosystem
Adaptability: The Evolving
We need to be
more driven by
This is where
What have we learned from
Best-of-breed, point solutions are challenging to operate
Redundant technology infrastructure
Multiple skill sets required
The fully-integrated platforms such Cerner and Epic are
“Meaningful use” of the technology is critically important
We are seeing the same patterns in analytics
Numerous fragmented point solutions, data quality problems
Producing reports but not applying the analytics to affect
quality and cost
Healthcare Analytics Adoption Model
& Prescriptive Analytics
Tailoring patient care based on population outcomes
and genetic data. Fee-for-quality rewards health
Clinical Risk Intervention
& Predictive Analytics
Organizational processes for intervention are
supported with predictive risk models. Fee-for-quality
includes fixed per capita payment.
Population Health Management
& Suggestive Analytics
Tailoring patient care based upon population metrics.
Fee-for-quality includes bundled per case payment.
Waste & Care Variability Reduction
Reducing variability in care processes. Focusing on
internal optimization and waste reduction.
Automated External Reporting
Efficient, consistent production of reports &
adaptability to changing requirements.
Automated Internal Reporting
Efficient, consistent production of reports &
widespread availability in the organization.
& Patient Registries
Relating and organizing the core data content.
Enterprise Data Warehouse
Collecting and integrating the core data content.
Fragmented Point Solutions
Inefficient, inconsistent versions of the truth.
Cumbersome internal and external reporting.
Progression in the Model
The patterns at each level
Data content expands
Data timeliness increases
To support faster decision cycles and lower “Mean Time To
Data governance expands
Adding new sources of data to expand our understanding of
care delivery and the patient
Advocating greater data access, utilization, and quality
The complexity of data binding and algorithms increases
From descriptive to prescriptive analytics
From “What happened?” to “What should we do?”
The Expanding Ecosystem of Data Content
1. Billing data
2. Lab data
3. Imaging data
4. Inpatient EMR data
5. Outpatient EMR data
6. Claims data
7. HIE data
8. Detailed cost accounting data*
9. Bedside monitoring data
10. External pharmacy data
11. Familial data
12. Home monitoring data
13. Patient reported outcomes data*
14. Long term care facility data
15. Genomic data
16. Real time 7x24 biometric monitoring
data for all patients in the ACO
* - Not currently being addressed by vendor products
Closed Loop Analytic
EMR, pharmacy, lab, imaging, RCM, materials
management, cost accounting
Care process algorithms
Triage criteria, order sets, protocols
Provider and patient education material
Patient and care management reports
Organizational data literacy
Process improvement training
Clinical leadership teams
Data & knowledge asset
Steering and guidance
Quality of Care vs. Cost of Care
Enterprise data warehouse
Data access & production
Deeper In the
2. Data Modeling & Analytic Logic
3. Master Reference/Master Data Management
7. EDW Performance and Utilization Metrics
8. Hardware and Software Infrastructure
EDW Performance Metrics
Hardware & Software Infrastructure
Key issues: Reliability, supportability, reuse
What tools does the solution use? Who owns the
What is the ETL design for updates? (Full,
Does the solution have a library of ETL
“accelerators” to common source systems?
Options, in order of preference
Kimball Dimensional Star Schema
Inmon Corporate Information Model
Bus architecture is rapidly adaptable and very flexible. It places more emphasis
on data marts that support specific analytic needs and scenarios, rather than a
general analytic model to support all analytic needs, especially those that are
focused on patient cohorts and registries.
Dimensional models have a very limited scope of usefulness in healthcare–
typically best suited for finance and materials management/supply chain
Purchasing an Enterprise Model might seem like a good idea, but the ETL is very
difficult to maintain; the model is not easily adaptable to new source systems; and
analysts prefer more specific models to suit their needs.
I2B2 is very specific to healthcare, particularly designed to support academic
medical centers, but it is very complex. Few people in the country understand it
and can support it, and its usefulness in meeting more typical analytic scenarios is
No single data modeling strategy will meet all analytic scenarios.
Data Mart Data Modeling
The data models are important, but the analytic
logic associated with the content of the data
marts and reporting is more important
High value logic
Data Models and
Does the solution support each layer? Prove it…
What is the vendor’s strategy?
Mandatory or voluntary compliance and mapping to master data
Mandatory compliance and mapping is unnecessary and can
lead to disaster
What data model and structures are used to support the
How does the vendor accommodate international, national,
regional, and local master data management?
Do they use an external vendor partner?
Do they support mappings to RxNorm, LOINC, SNOMED, ICD,
Do they support a user-friend interface terminology?
Can you browse and search metadata from a
Does the solution require an expensive add-on
Does it collect metadata from ETL jobs and the
Does it allow a “wiki” style contribution of
Is there a bundled, preferred visualization tool?
Is it affordable and extensible if exposed to all
employees and patients?
Is the data model(s) decoupled from the
Does the data model support multiple
visualization tools and delivery of data content?
Are there fewer than 20 roles in the initial deployment?
Does the solution employ database level security, visualization layer
security, or some combination of both?
Does the vendor’s security philosophy pass the test for maintainability?
Does it balance security with access?
How does it handle patient identifiable data?
How does the security model manage access to extremely sensitive
personal health information, such as behavioral health, AIDS, etc.?
How does is handle physician identifiable data?
What type of tools and reports are available for managing security?
Can the tools identify “unusual” behavior, such as repeated mass
downloads of data?
EDW Performance and
Can the solution track basic data about the
environment, such as:
User access patterns
Query response times
Data access patterns
Volumes of data
Hardware and Software
Oracle, Microsoft, IBM are the only realistic options
Microsoft is the most integrated, easy to manage,
and affordable… from database management
through analytic desktop
Scalability is no longer an issue– it scales to multiterabyte databases, easily
Windows is viable and can compete with Unix in all
but the largest clusters…years away, if ever, for
most healthcare organizations
IBM is a good second choice, but has a small
Oracle is expensive and lacks integrated tools
Knowledge and Deployment Systems
Does the vendor support closed loop analytics that bends analytic
knowledge back to the point of care and/or workflow?
What do their customers say about their ability to improve care and
Have they had experience with actually realizing an ROI from the
What are the success stories-- where quality of care improved? Cost
of care decreased?
What tools and processes does the solution have to support:
Continuous quality improvement and cultural change initiatives?
Cost control initiatives?
Activity based costing?
Prioritization of analytic efforts and improvement programs?
What tools or experience does the solution offer for data governance?
Clinical Content and
Evidence Based Analytics
Does the solution leverage evidence based
clinical content in the design?
Data model, patient registries, benchmarking
Are the analytics on the back end integrated
with evidence based data collection on the front
end, such as order sets and clinical guidelines?
Can the system measure adherence to clinical
evidence and guidelines?
Timelines and Costs
Can the solution offer business value in less than 3
months, in constant increments?
Does the solution cost less than $7M over three
years for a $1B - 2B organization (scale up and
Vendors in the Crowded Market
Health Care Dataworks
Strata Decision Technology
White Cloud Analytics
The analytic environment in healthcare is rapidly
changing, and that’s not going to stop
Adaptability of the technology is crucial
Technology is only 1/3 of the solution
Cultural willingness to embrace analytics is crucial
Cultural processes for sustained implementation are
Look for a vendor that offers a total solution– closed