Julia Adler-Milstein's keynote presentation at the University of Michigan LHS Collaboratory, March 14, 2017.

1. +
The Missing Piece?
Understanding Provider Organization Capabilities to Engage
with the Learning Health System
Julia Adler-Milstein, PhD
March 14, 2017

2. +
Overview of Talk
 Setting the Context: Lessons from Other Industries on IT
Value
 Knowledge Management as a Provider Organization
Competency
 Knowledge in the Era of Health Data Science
 Implications for Provider Organizations and LHS
Engagement

3. Lessons on Realizing Tech
Value from Other Industries
David P. (1990) The Dynamo and The Computer.

4. Lessons on Realizing IT
Value from Other Industries

5. 5
“New technology takes time to have
a big economic impact.
More importantly, businesses […]
have to adapt before that will
happen.”
http://www.slate.com/articles/arts/the_undercover_economist/2007/06/the_shock_of_the_new.html

6. Organizational Context
IT
Outcomes
 Organizational contexts conducive to realizing IT value:
o give frontline staff the authority and accountability to make
decisions based on newly available, real-time data,
o and ensure they have the training and skills to do so
Lessons on Realizing IT
Value from Other Industries
e.g., Brynjolfsson E, Hitt L. (1996). Firm-level Evidence on the Returns to Information Systems Spending. Management Science.

7. Healthcare

8. The EHR-Performance Gap
 On the one hand…
o Early studies from individual institutions reveal substantial quality and
efficiency gains from EHRs
 Served as the motivation for HITECH
 On the other hand…
o Recent, large-scale studies fail to find a consistent relationship between
EHR adoption and improved performance (e.g., Appari 2012, Adler-Milstein et al. 2013)

9. +
 MAGICAL THINKING
 REALITY
IT Better
Performance
IT Better
PerformanceComplementary
Organizational
Changes

10. +
Overview of Talk
 Setting the Context: Lessons from Other Industries on IT
Value
 Knowledge Management as a Provider Organization
Competency
 Knowledge in the Era of Health Data Science
 Implications for Provider Organizations and LHS
Engagement

11. +
What is knowledge? What is
knowledge management?
 In the context of this talk…
 Knowledge about health
and healthcare that is
generated outside the
practice setting (provider
organization)
 Knowledge management
is the dynamic set of
organizational capabilities
needed to convert
knowledge to practice

12. +
Current State
 Lack consensus on KM competencies
 And how they need to adapt as external knowledge and
knowledge dissemination infrastructures change
 Lack data on current state of KM in US provider
organizations
 Can look to leading provider organizations for
examples

13. +
Knowledge Management: Strategic Goals
 Reduce the cost and increase the speed of
knowledge acquisition and maintenance for decision
support
 Speed translation of clinical innovation and evidence
into clinical practice
 Proactive, anticipatory decision support architecture
 Improve organizational effectiveness as a learning
organization through organizational alignment and
data-driven performance improvement
Knowledge management as decision support
Adapted from T. Hongsermeier

14. +
Content Life-Cycle Challenges:
Committee, Department,
Researcher, or Other
Proposes to Implement Content
Guideline is Defined and Validated
Functional Knowledge Specification
For Encoding is
Designed and Validated
Ongoing Revisions or
Eventual Sunset
Of Encoded Guideline
•Prioritization mechanism not always clear
•Stewardship processes not always clear
•Lack of coordination
•Unclear mechanism for subject matter expert participation…
•No budgetary model to reimburse experts…
•No tools to support efficient collaboration
•Little or no audit trail of decisions made
•Project competition with other engineering projects,
prioritization processes unclear
•Knowledge editors typically do not enable content
auditing, knowledge editors siloed, no support of
inheritance or propagation
•Little or no documentation about content in production
•MS Office doesn’t help maintain data about content
•Little analytic data available on decision support content or
impact on clinical outcomes impact to direct updating
•Tendency to rely on query of transaction systems
•No content management tools to support process and
ensure timeliness
Specification is
Engineered into Production Generating
a Technical Specification
Adapted from T. Hongsermeier

15. +
Evolving knowledge management infrastructure:
Knowledge Management Generation 1:
• Build and deploy a document library to provide enterprise wide access
to specifications of decision support knowledge
• Inventory all structured knowledge in production
• Create and develop a knowledge repository
Knowledge Management Generation 2:
• Implement tools to support collaborative content consensus, iterative
drafting of guidelines and conversion to functional knowledge
specifications
• Knowledge repository expanded to support browsing of pre-production
and “in-production” knowledge
• Implement tools to support content management processes using
lifecycles and workflows (knowledge maintenance)
Knowledge Management Generation 3:
• Integrate legacy and new content authoring tools with content
management infrastructure (knowledge editing)
Adapted from T. Hongsermeier

16. +
Current State
 KM not a recognized “competency” of provider
organizations
 Where KM is occurring, it is mostly focused on:
 Proving access to external information resources
 Deciding what should decision support should include
 Substantial disparities in KM capabilities by type of
provider organization

17. +
Two observations
 The huge investment in 21st century health
knowledge generation has not been coupled with
investment in 21st century knowledge
application.
 It is critical to anticipate how knowledge will
change, and how healthcare delivery
organizations will need to adapt.

18. +
Overview of Talk
 Setting the Context: Lessons from Other Industries on IT
Value
 Knowledge Management as a Provider Organization
Competency
 Knowledge in the Era of Health Data Science
 Implications for Provider Organizations and LHS
Engagement

19. +
Knowledge will:
Come in more forms and at different levels of scale (i.e.,
individual, population)
Be regularly changing and updating
Be inherently probabilistic
Be customizable to specific people and situations
Come via more channels (i.e., beyond journals and
guidelines)
Come from a variety of sources
Be more accessible
Be a recognized “entity”

20. +
Overview of Talk
 Setting the Context: Lessons from Other Industries on IT
Value
 Knowledge Management as a Provider Organization
Competency
 Knowledge in the Era of Health Data Science
 Implications for Provider Organizations and LHS
Engagement

21. +
Knowledge will: Which will require
healthcare delivery
organizations to:
Come in more forms
and at different levels
of scale (i.e.,
individual,
population)
Have a process for
“local” translation and
operationalization

22. +
Knowledge will: Which will require
healthcare delivery
organizations to:
Be regularly
changing and
updating
Have a rapid process
for decision-making
about fit/relevance
Have frontline work
processes that can
continuously adapt

23. +
Knowledge will: Which will require
healthcare delivery
organizations to:
Be inherently
probabilistic
Have a workforce that
can make decisions
under conditions of
uncertainty

24. +
Knowledge will: Which will require
healthcare delivery
organizations to:
Be customizable to
specific people and
situations
Have infrastructure for
mass customization

25. +
Knowledge will: Which will require
healthcare delivery
organizations to:
Come via more
channels (i.e.,
beyond journals and
guidelines)
Have varied
mechanisms of receipt

26. +
Knowledge will: Which will require
healthcare delivery
organizations to:
Come from a variety
of sources
Have a process to
validate and trust

27. +
Knowledge will: Which will require
healthcare delivery
organizations to:
Be more accessible Have a workforce and
work processes that
enable direct access to
role-relevant
knowledge

28. +
Knowledge will: Which will require
healthcare delivery
organizations to:
Be a recognized
“entity”
Have a governance
process for adopted
knowledge

29. +
Knowledge will: Which will require provider
organizations to:
Come in more forms and at different levels
of scale (i.e., individual, population)
Have a process for “local” translation and
operationalization
Be regularly changing and updating Have a rapid process for decision-making about
fit/relevance
Have frontline work processes that can
continuously adapt
Be inherently probabilistic Have a workforce that can make decisions
under conditions of uncertainty
Be customizable to specific people and
situations
Have infrastructure for mass customization
Come via more channels (i.e., beyond
journals and guidelines)
Have varied mechanisms of receipt
Come from a variety of sources Have a process to validate and trust
Be more accessible Have a workforce and work processes that
enable direct access to role-relevant knowledge
Be a recognized “entity” Have a governance process for adopted
knowledge

30. +
What do we know about how to do
these things well?
Tidbits like…
 Healthcare delivery organizations will need a workforce that
includes those with familiar titles (e.g., “doctor” and “nurse”) but
new skillsets, alongside people with entirely new roles.
 These new skillsets and roles are beginning to be conceptualized
and articulated in the form of competencies that include: (1)
knowing what you do and don’t know, (2) ability to ask a good
question, and (3) skills in evaluating and weighing evidence.
 Yet there is no guidance for healthcare delivery organizations in
terms of how to increase these competencies in their workforce.

31. +
What do we know about how to do
these things well?
Literature from other industries that are ahead of
healthcare in their data science maturity
 Point to the need to put information and the relevant decision
rights in the same location.
 Specifically, when information is created and transferred, and expertise
is often not where it used to be, an organization needs to be flexible
enough to minimize the “not invented here” syndrome and maximize
cross-functional cooperation.
 Also a need to shift the culture of an organization to one in which
the first question is not “What do we think?” but “What do we
know?” as well as “Where did the data come from?”, “What kinds
of analyses were conducted?” and “How confident are we in the
results?”

32. +
Concluding Thoughts
Nascent state of understanding about
 how knowledge characteristics will change,
 how healthcare delivery organizations will need to
change in response,
 how to execute those changes
Such understanding is necessary to bridge the
“last mile” of the learning health system.
It is therefore imperative to begin the process of
discovery.