2. Using Zoom
Turn your webcam on!
Please remember to mute
yourself during the presentations.
If you have a question, you may
un-mute yourself and ask after
each presentation, OR use the
Q&A button
3. Advanced Team Based Care (TBC)
Learning Collaborative
Welcome to Session 3
December 14, 2016
3:00 - 4:30 EST
Developing core and extended team capabilities for implementing an advanced team based care model.
4. Introductions
TBC Collaborative Design, Facilitation, Faculty
Ann Marie R Hess ANP, MS
National Cooperative Agreement
Anna Rogers, Director
Reema Mistry, Program Coordinator
Mentors , Coaching Faculty
Deborah Ward, RN (1:8)
Kasey Harding (1:8)
Evaluation Faculty
Kathleen Thies, PhD, RN
Improvement Science Faculty
Patti Feeney
Mark Splaine, MD
5. Objectives Session 3
Summarize Action Period 2 Milestones (6 weeks)
Learn how to use data for improvement
Learn from team Specific Aims and PDSAs
→Healthcare for the Homeless
→Carolina Family Health Center
→The Children's Clinic
Provide path forward and resources for Action Period 3
6. Session 2 Feedback
88%-100%
Visuals supported TBC content
Session met learning needs
Can apply information to our practice
Learned a moderate to great amount
Teaching methods were effective
Improvements
o Provide more examples of PDSA cycles and fishbone diagrams
o Send agenda and next assignments ahead of time
o Provide more examples of what it means to create a team environment
o We would like more statistics about what works and what does not work
o At the time of day for us, people tired even though sessions very educational
7.
8. 16 Teams : most teams adjusted core and extended
93 participating Core and Extended Team Members
Interact daily/weekly with patients and families
Roles: MD, NP, LPN, BH, Care Coordinator, CNA, FNP, Care Manager, MA, NP,
Front Desk, RN, LVN, Radiology Tech, LCSW, PNP, Interpreter, Case Manager,
Dental Coordinator, Pharmacist, Call Center Rep, Outreach Specialist, MSW
POD structure with multiple providers and dedicated MAs (4)
Other Leadership and Management : CQO, Quality Coord, Dir of Ops, Site Manager,
HR Dir, Data Analyst, Ops Specialist, Clinical Support Services Manager, Compliance,
COO, CMO, EMR Manager, Clinical Manager, IT, Patient Financial Coord, Dir Patient
Services, BH Manager, Medical Director, Chief Clinical Officer, CMO, Referral Manager
13. Agenda (3:00-4:30)
3:00 5 min Welcome and Introductions
Putting it All Together : Improvement Ramp
3:05 10 min Action Period 2 Milestones and Challenges
3:15 35 min Using Data for Improvement
3:50 30 min TBC Improvement Work
-Healthcare for the Homeless Team
-Carolina Family Health Center Team
-The Children’s Clinic
4:20 10 min Path Forward and Resources [ Introduce Progress Check List]
Action Period 3 Assignments
14. Improvement Science Theory Bursts (10 min)
Developing Capacity for Implementing Advanced TBC Model
Session 1 : Sept 21st
o Running effective team meetings using tools
o Developing and using a cause and effect diagram to inform PDSAs
o Writing a global and specific aim statement
Session 2 : Nov 2nd
o Developing a process map or current state workflow
o Applying PDSA methodology for improvement
Session 3 : Dec 14
o Using data for improvement (run charts, bar graphs, sampling)
Session 4 : Jan 25
o Standardizing (SDSAs) and Reliability Science
Session 5 : March 15
o Spreading Change
Session 6 : April 26
o Gantt Charting : 3-6 month Core Team improvement plan
15. Define Core and
Extended Team
Achieve multiple TBC
specific aims
Standardize (SDSAs)
roles and key processes
(Playbook, Spread Plan)
Improve team and coach
skills (improvement
science, team work,
coaching)
Move Practice
Assessment Data toward
Level A
Develop a post
collaborative team
action plan
TBC Learning Collaborative
90 min
Learning
Sessions
Between Session Action Periods (6 weeks)
Complete Assignments
Weekly Team Meetings , Daily Huddles
Monthly Reporting
Share Your Work – TBC Website (Moodle)
Developing Effective Meeting and Improvement Skills
Implementing Team Based Care – Small Tests of Change
Learning from Each Other
Action Period 3
Core and Extended Team
Refinements –
challenges
TBC Webinars
Effective Meetings and
Daily Huddles
Readiness Survey
Role Activity and Cycle
Time Data - deeper
Global, Specific Aims
Fishbone : Defining
Problem and PDSAs
Process Mapping :
Workflow and Roles
Brainstorming and
Benchmarking
PDSAs
SDSAs
Between Session
Mentoring and Faculty Support
Moodle Resources and Discussion Board
7
Sept 21 Dec 14 Jan 25 Mar 15 Apr 26 June 14Nov 2
16. Mentors Helping
Weekly Coach : Mentor Group Meetings, Individual as Needed
Teaching skills, reviewing tools (more skill building
needed from theory bursts, struggling with tools)
Addressing Online Learning Network Site (Moodle)
challenges
Advising difficulties getting meetings off the ground
(no time to meet), using roles (resistance)
Clarifying assignments, how to use assessment
tools
Providing advice for managing : ‘turmoil’,
‘overwhelming assignments’, ‘team and leadership
engagement issues’ , ‘team vs coach ownership’,
‘worried management will roadblock us’
Reminding : Start Where you Are, Use What you
Have, Do What you Can….
Weekly Meetings : Action Period 3
*no meeting tomorrow 3pm
Rate the Meetings
On average 8-9 (scale of 1 -10) .
Most helpful:
hearing report outs from other teams and
progress they are making,
reinforcing and learning new skills to help our
teams,
learning from how others are overcoming
challenges,
staying on track with assignments and getting
help with how to complete
getting help with how to use data
17. Action Period 2
Milestones
More than half the teams have been:
Refining their core and extended team structures
Defining roles and communication – both within
core team, and between core and extended team
Challenges:
• Keeping same staff in Core Team or a POD
• Staff pulled for coverage other shifts or sites
• Significant turnover (Providers, MAs, RNs)
• RNs more attracted to extended team role
• Other team members who cover us do not know
what we are doing…
18. Action Period 2 : Give Teams Time to Meet (webinar 1)
Challenges:
Time to meet. Getting time when ‘necessary people’ can get there.
Team members not engaged
Turnover – significant. Staff pulled for coverage other sites, departments
Provider not willing to delegate, when ultimately responsible if does not get done
Leadership support (e.g.’buy in’ to the process, communication about the work)
10 Teams (overcoming challenges)
working hard at sustaining 3 meetings/month, some 4-5/ month), 45-60 min
finding a good time to meet after trials of different times
learning who is required to attend – those needed to do the work, invite others when needed
meeting even if a few people (discipline and rhythm), start on time and do not wait for people
getting leadership support for protected time
sending meeting reminders (e.g. text 15 min ahead of meeting)
engaging team members
o using fishbone and process mapping (‘have stake in it’)
o rotating roles, assigning timekeeper for easy role
o posting work on the wall for input between meetings
6 Teams : meeting 2 or fewer times/month, working on unique challenges, different pace
20. Aim: Reduce waiting room time from 25 min
to 14 min By Dec 30
Weekly Data
5 Patients
To Date: 19 min from 25 min
Adding observation by Coach: MA Rooming
PDSAs :
Redesign and standardize MA Rooming Process
– more time with MA (value added)
Team Engagement
Peach Tree Healthcare
Action Period 2
Example : Data, Mapping, Aims, PDSAs
21. Early Stages of Change MA Role, RN Just Getting Started
(Use Your Data)
Peach Tree Healthcare
22. Holyoke Health Center
MA Role
RN Role
Insights 8 hours of Tracking
Significant Duplication of Efforts
Double Documentation (paper, EHR)
No Standards and Protocols for activities
‘Insufficient’ RN Care Management and
Coordination time – 16 min
Challenging Questions
What is ideal time for advanced activity?
All roles at once?
How do you narrow down roles and
activity to work on?
24. Role Activity Challenges (Action Period 3)
Common Challenges
Provider Completion of Role Activity
More differences than we anticipated
between what we are ‘currently doing’
and what we identified as ‘ideal’
Fears about giving things up, adding new
Lack of role delineation between LPN
and RN – and Provider duplication
Variation between same roles (2-3 MAs)
So many inefficiencies to tackle
Some Opportunities
Shadowing provider half day - activity
tracking. Shorter periods over days.
Activity analyses helping improve job
descriptions, role delineation, optimization
Transparency of work opportunity to ask –
We are doing ‘what’? We are putting it
‘where’? Why are we doing ‘that’?
Eliminating duplication of documentation
Reducing interruptions by optimizing EHR
messaging, workflows
25. Using Fishbone Long Cycle Time
to Identify Aims and PDSAs (Action Period 3)
Key Drivers
Pre Registered vs Not
On site lab delays
Early and late arrivals (team on time?)
AIM : Increase Percentage of Patients that
are Pre Registered from 26% to 29%
by Dec 31st
Team Engagement
High Leverage PDSAs
(Action Period 3)
Efficiency, Role Optimization
AM , PM Start Times
Rooming Standards
Daily Huddles (6 Teams)
Pre Visit Planning
Sumter Family Health Center
26. PDSA Discipline (Action Period 3)
Increase Complexity of Change
- Start Small Test, days –weeks
- Small Wins
- Each PDSA can have a measure
Increase number of patients
- Have standards and protocols
- Have standard workflows
- Hard to implement
Get it right , Fewer Patients- before
scaling up to a defined population
27. Daughters of Charity
80
0
10
20
30
40
50
60
70
80
90
October 2016
Total Cycle Time - Check In to Check Out
Use Your Data (Cycle Time) Track Over Time (Action Period 3)
(5 / week, 15/ month)
How long do we track it? Achieving results you wanted, sustaining results due
to standardization and process reliability, new habits in daily work
0
5
10
15
20
25
30
Check In
Time
Waiting
Room Time
Support
Staff Time
Exam Room
Wait Time
Time with
Provider
Check out
Time
Break Down of Cycle Time
Specific Aims
Goal: 30 min for 20 min appt type
28. Using Data for Improvement (Action Period 3)
66
86
76
68
91
0
10
20
30
40
50
60
70
80
90
100
Gathering and
Using Data Skills
Using Effective
Meeting Skills
Implementing
Daily Huddles
Using
Improvement
Skills
Applying
Teamwork Skills
%StronglyAgreeorAgree
Team Skills Self-Assessment Summary (N=68)
Percentage Strongly Agree or Agree are Competent
October 2016
30. Session on Data & Sampling
for the Team-Based Care Collaborative
Mark E. Splaine, MD, MS
December 14, 2016
31. Displaying Data & Sampling
• Three theory bursts
– Displaying data over time (5 minutes)
– Types of variation (5 minutes)
– Overview of run charts (5 minutes)
• Application exercise
– Interpreting a run chart example (10 minutes)
• Sampling for improvement work (5 mins)
• Questions and discussion (5 mins)
31
33. Time 1 October 26, 2016 33
0
20
40
60
80
100
120
140
160
180
total time in
office
check in time waiting room
time
time with the
nurse
exam room
wait time
time with
provider
check out
time
Mean Minimum Maximum
Cycle Time: Johnson City CHC
37. Time Plot
• A graph of data in time order
• Often kept to identify if and when problems
appear (proactive)
• Also used to see trends over time (reflection)
• Especially helpful when you implement a change
to follow the result
37
38. 60
80
100
120
140
160
180
200
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
Diabetes Stratified Time Plot
Days
BloodSugar(mg/dl)
Morning Readings
Before Bed Readings
38
Goal
40. Two Types of Variation
• Random (common cause) variation
• Non-random (special cause) variation
40
41. Random Variation
• Typically due to a large number of small
sources of variation
– Example: Variation in arrival time of a patient
might include: weather, vehicle problems,
parking issues
• Usually requires a deep understanding of
the process to change
41
42. Non-Random Variation
• Are not part of the process all the time.
Arise from special circumstances
– Example: Patients arrive late for appointments
due to a bus strike
• Usually best uncovered when monitoring
data in real time (or close to that)
42
43. How to React to Variation
ActionProcess result
Process with
only random
variation
Not satisfied with result:
redesign process to get a better
result
Reduce variation:
make the process even more
predictable or reliable
Process
with non-
random
variation
Identify the cause:
If positive, then can it be
replicated or standardized.
If negative, then cause needs to
be eliminated
Target the special causes - to
get the process predictable
43
47. Non-Random Patterns
on Run Charts
• The presence of a shift in the process
– A “run” is one or more consecutive points on the same
side of the median
– A run that is too long (6 or more consecutive points on
one side of the median)
• The presence of a trend
– A run with consecutive increases or decreases in data
(5 or more consecutive points)
• The presence of too much or too little variability
– Too few or too many runs (depends on number of
points on the chart)
47Perla, Provost, and Murray. BMJ Qual Saf. 2011;20:46-51
48. Source: Perla, Provost, and Murray. BMJ Qual Saf. 2011;20:46-51
48
Table. Runs Rule Guidance
Number of observations excluding points
on the median
Lower limit for the number of
runs
Upper limit for the number
of runs
13 4 11
14 4 12
15 5 12
16 5 13
17 5 13
18 6 14
19 6 15
20 6 16
21 7 16
22 7 17
23 7 17
24 8 18
25 8 18
26 9 19
27 10 19
28 10 20
29 10 20
30 11 21
31 11 22
49. 60
80
100
120
140
160
180
200
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
A Run is a point or group of consecutive points that fall
on one side of the median
Days
How to Count Runs
49
FastingBloodSugar(mg/dl)
50. Questions
1. What does the blue line on the graph
represent?
2. How many runs are there?
3. How many shifts do you see?
4. How many trends are in the data?
5. How many non-random patterns (special
cause signals) are met in this run chart?
6. What is your interpretation of the chart?
50
52. 60
80
100
120
140
160
180
200
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
Q1. What does the blue line represent?
Days
Run charts use the Median as the
central tendency measure
The Median
52
FastingBloodSugar(mg/dl)
53. 60
80
100
120
140
160
180
200
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
There are 14 Runs
Days
Q2. How many runs are there?
53
FastingBloodSugar(mg/dl)
54. 60
80
100
120
140
160
180
200
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
There is ONE Shift
Days
Q3. How many shifts do you see?
54
FastingBloodSugar(mg/dl)
55. 60
80
100
120
140
160
180
200
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
There are NO Trends
Days
Q4. How many trends are in the data?
55
FastingBloodSugar(mg/dl)
56. 60
80
100
120
140
160
180
200
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
Runs = 14 Shifts = 1 Trends = 0
There is ONE Signal
Days
Non-random pattern (Shift)
Q5. How many non-random patterns?
56
FastingBloodSugar(mg/dl)
57. 60
80
100
120
140
160
180
200
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
• There is non-random (special cause) variation present. One would need
to investigate why this occurred. Since the cause is in the wrong
direction, one would ideally like to eliminate this cause from the system.
• Note: upon talking to the patient, the special cause was related to him
eating dessert every night while on vacation. Some education about diet
could then eliminate the cause.
Days
Non-random pattern (Shift)
Q6. What is your interpretation of the chart?
57
FastingBloodSugar(mg/dl)
59. Two Type of Sampling
Random (Probability)
Samples
• Think of a pond or lake
• Water stays in place
Judgment Samples
• Think of a stream or river
• Water constantly moving
59
60. Improvement Work
• Benefits from judgment sampling
• What is judgment sampling?
– A nonprobability sample that is selected on the basis of
knowledge of the process or a subject matter expert
• Is there a trade-off to using judgment sampling?
– “We trade the ability to quantify the precision of estimation
and control the bias of selection of a defined population for
learning about variation in the fragments of experience we
are most interested in learning about – most often with an
eye toward efficiency and getting ‘just enough’ data to guide
our learning and subsequent testing” 60
61. Some Examples
61
Example Situation Probability Sample Judgment Sample
5
Nurse leader wants
to test impact of
new pressure ulcer
bundle
• Obtain list of all units
• Randomly select 50%
• Assess all patients on
selected units before
and once after
intervention
• Sample 5 patients each
week who are at highest
risk on the unit with the
highest risk patients
• Track data over time
6
Oncology manager
wants to know
whether patients
get proper
education after
flowsheet
initiation
• Simple random sample
of all patients in last 3
months
• Charts reviewed by
manager
• Select the most recent
patients
• Perform chart review
62. Summary
• Variation over time is intrinsic to all health care &
other work processes.
• Displaying data over time can help visualize the
variation present.
• Understanding that variation can help monitor,
adjust and improve processes.
• Studying variation with run charts can offer
insights about possible cause of that variation and
offer clues to the design of change.
62
63. What haven’t we figured out yet?
Questions or issues that remain unclear?
63
64. References
• Perla RJ, Provost LP, and Murray SK. The run chart: a
simple analytical tool for learning from variation in
healthcare processes. BMJ Qual Saf. 2011;20:46-51.
• Perla RJ and Provost LP. Judgment sampling: a healthcare
improvement perspective. Qual Manage Health Care.
2012;21(3):169-175.
• George ML, Rowlands D, Price M, and Maxley J. The Lean
Six Sigma Pocket Toolbook. New York, NY: McGraw-Hill,
2005. Chapters 6 and 7, pp 104-118.
65. Agenda (3:00-4:30)
3:00 5 min Welcome and Introductions
Putting it All Together : Improvement Ramp
3:05 10 min Action Period 2 Milestones and Challenges
3:15 35 min Using Data for Improvement
3:50 30 min TBC Improvement Work
-Healthcare for the Homeless Team
-Carolina Family Health Center Team
-The Children’s Clinic
4:20 10 min Path Forward and Resources [ Introduce Progress Check List]
Action Period 3 Assignments
66. Healthcare For The Homeless
Specific Aim and PDSAs
Referral Process : RN Role
Optimization Extended Team
67.
68. Manages Referrals to specialists
and community resources,
ensuring relevant clinical
information is provided
B C
72. Carolina Family Health Centers
Specific Aim : Cycle Time
Pre Visit Planning Daily Huddles
73.
74. 0
10
20
30
40
50
60
70
80
90
Total Cycle
Time -
Check In to
Check Out
Check In
Time
Waiting
Room Time
Support
Staff Time
Exam Room
Wait Time
Time with
Provider
Time at Lab Time at
Referrals
Check out
Time
Oct-16
Oct-16
Cycle Time : Average 78 minutes
81. Huddles
• What time of day, and how long are your huddles?
• What is the focus of your huddles?
• Are you using a tool?
• How are you including your Care Coordinator?
• How are you thinking about including your Patient
Service Representative?
84. Agenda (3:00-4:30)
3:00 5 min Welcome and Introductions
Putting it All Together : Improvement Ramp
3:05 10 min Action Period 2 Milestones and Challenges
3:15 35 min Using Data for Improvement
3:50 30 min TBC Improvement Work
-Healthcare for the Homeless Team
-Carolina Family Health Center Team
-The Children’s Clinic
4:20 10 min Path Forward and Resources [ Introduce Progress Check List]
Action Period 3 Assignments
86. Action Period 3 Assignments
1. Meet weekly as a Core Team
Problem solve ‘time to meet’
Practice effective meeting skills using tools, with coaching support
Define core and extended care team (members and roles)
2. Implement Daily Huddles
Work on improving (PDSAs) and standardizing (SDSAs)
Align Huddle intervention with a specific aim (e.g. reduce cycle time, increase screening)
3. Write specific aim(s) statements , using data and knowledge of problem
Continue Assessments (role activity, cycle time, other)
Complete Fishbone diagrams and process mapping
87. Action Period 3 Assignments
4. Implement PDSAs (small, measurable, rapid)
Share your work by uploading TBC website, discussion board
Use brainstorming and benchmarking to inform changes
5. Complete readiness survey ( 50 % ), data will be posted on TBD Website
Purpose : To assess whether an organization is ready and committed to the
implementation of a specific change, from the perspective of care team members.
Still time to complete Team Skills Assessment survey (50%)
6. Post Monthly Reports : Next Due January 10th , 2017
7. Watch Webinars (1-4)
8. Introduce Coaches to Progress Check List for tracking implementation, leadership reporting