Systems modeling provides the evaluator with a method to empirically explore potential impact of an intervention and, in particular, the economics of features and benefits of that intervention. Assumptions that support the intervention can be tested and the varying perspectives of multiple stakeholders can be systematically evaluated as a means to both quantify the effects of their unique perspectives and also to promote consensus building and dialogue among stakeholders that is grounded in empirical evidence. This session will review the work done in a medical center to model the economics of required training programs for new employees. The specific aim was to identify the optimum timing for compliance based training utilizing economic considerations and this model as the basis for beginning discussions of policy changes within the organization. Additionally modeling the tradeoffs between optimizing economic variables and optimizing non-economic variables such as learning will be discussed.

1. Evaluating opportunities to
optimize learning and economic
impact:
Applying system dynamics to
model training deployment in a
medical center
Daniel McLinden, EdD
Assistant Vice President
Education & Learning Department
Cincinnati Children’s Hospital Medical Center
Assistant Professor
Department of Pediatrics
University of Cincinnati College of Medicine
Stacey Farber, PhD
Manager, Education Research and Measurement
Education & Learning Department
Cincinnati Children’s Hospital Medical Center

2. Our Vision: To be the leader in improving child health
Historical Highlights
• Established in 1883
• Development of the Sabin oral polio vaccine
• Creation of the first heart-lung machine
• Rotavirus vaccine developed and tested by
Children's researcher
National Recognition
• U.S. News & World Report (2008) ranked Cincinnati
Children’s Hospital Medical Center (CCHMC) the
third best children’s hospital in the United
States
• Child magazine named CCHMC one of the top five
best children's hospital in the United States by.
In addition, cancer care and neonatal care at
Cincinnati Children's were named among the top
five subspecialty programs in the United States.
• Recipient of the 2006 American Hospital
Association-McKesson Quest for Quality Prize™
honoring leadership and innovation in quality, safety
and commitment to patient care
Full-service, not-for-
profit pediatric
academic medical
center with 475
registered inpatient
beds and 36 registered
residential psychiatry
beds

3. • 10,000+ person organization
• 1000+ new hires each year
• Multiple training requirements driven by
external agencies
• Risk associated with training compliance
• Cost association with training compliance
The situation

4. Engage Stakeholders to make explicit the
features, assumptions and outcomes.
Improved employee morale will
make this a better place to work
but will not affect our business
either positively or negatively.
The bottom line will
improve if we improve
employee morale and this
program is just what we
need to increase morale.

5. A logic model is…
• A tool for telling your program’s story
– McLaughlin, J.A. & Jordan, G.B. (1999). Logic models: A tool
for telling your program's performance story. Evaluation and
Program Planning, 22, 65-72.
• A connection of theory with implementation and
outcomes
– Frechtling, J.A. (2007). Logic Modeling in Program Evaluation,
San Francisco: Jossey Bass.
• A tool to enable collaborative planning for
program impact
– Kraigier, K., McLinden, D., & Casper, W.J. (2004). Collaborative
planning for training impact. Human Resource Management,
43(4), 337-351.

6. Required Training – Logic model
Newly hired
staff
Scheduling of
programs
OSHA
requirements met
Requirements for
Institutional
initiatives met
Instructors for
classroom
programs
INPUTS OUTCOMES
Joint Commision
requirements met
Organization
is compliant
with
requirements,
Personnel are
aware of
norms and
expectations
of them.
Etc.
OUTPUTS
Completion of
required training
Context
• Participants are required to locate course in Learning Management
System
• Participants are required to manage their compliance
• Participants can enroll at any time during and after the required timing
New employees are
aware of 30 day
requirements
New employees are
aware of 365 day
requirements

7. What are the Economics: A Spreadsheet model
CARES Orientation
Discrimination and Harassment Awareness in the
Workplace
Collaborative IRB Training Initiative (CITI)
Diversity Orientation (Step 1)
Diversity Awareness (Step 2)
Diversity for Managers
HIPAA Training
New Employee Orientation (NEO)
New Employee Safety Test
NIMS for NEO
NIMS HICS 100
2007 Patient Safety Education
Safety College
Etc.
What is the cost of
participant time
Training time
Number of training
participants

8. What are the economics: A system model
• System models simulate change over time in a
system.
• Incorporate nonlinearities, feedback, and
delays.
• Define the linkages between cause and effect
to test assumptions.
• Discover how to adjust cause and effect
relationships to improve outcomes
• Quickly simulate multiple scenarios to test new
assumptions
• Offer and experiential way to share the logic

9. Steps in the modeling process*
*Excerpted from Sterman, J.D. (2000). Business Dynamics: Systems Thinking and
Modeling for a Complex World. Boston: Irwin McGraw-Hill, (p.86).
2. Dynamic
Hypothesis
1. Problem Definition
3. Formulation
of model
4. Testing
5. Policy
Formulation

10. What are the implications of the current
approach to required training
2. Dynamic
Hypothesis
1. Problem
Definition
3.
Formulation
of model
4. Testing
5. Policy
Formulation
1,000+
• Multiple program requirements
• Multiple time requirements
• Self selection

11. What are the implications of the current
approach to required training
Start End of year one
1,000+
Self selection
2. Dynamic
Hypothesis
1. Problem
Definition
3.
Formulation
of model
4. Testing
5. Policy
Formulation

12. Change the model of required training
2. Dynamic
Hypothesis
1. Problem Definition
3.
Formulation
of model
4. Testing
5. Policy
Formulation
Start End of year one
1,000+
Orient Onboard Optimize

13. Cost is constant (almost)
Productivity is variable
Start End of year one
1,000+
Orient Onboard Optimize
$ $ $
Cost
2. Dynamic
Hypothesis
1. Problem Definition
3.
Formulation
of model
4. Testing
5. Policy
Formulation

14. 2. Dynamic
Hypothesis
1. Problem Definition
3.
Formulation
of model
4. Testing
5. Policy
Formulation
Assume time to competence is not linear

15. 2. Dynamic
Hypothesis
1. Problem Definition
3.
Formulation
of model
4. Testing
5. Policy
Formulation
Orient Onboard Optimize
New employees transit through the growth curve

16. 2. Dynamic
Hypothesis
1. Problem Definition
3.
Formulation
of model
4. Testing
5. Policy
Formulation
Orient Onboard Optimize
Value is a function of productivity relative to cost
$

17. Stocks and Flows
2. Dynamic
Hypothesis
1. Problem Definition
3. Formulation
of model
4. Testing
5. Policy
Formulation
1,000+ Orient Onboard Optimize
100 1000
+
Month 1 Months 2 -10 Months 11-12

18. Convert time and people to cost
2. Dynamic
Hypothesis
1. Problem Definition
3. Formulation
of model
4. Testing
5. Policy
Formulation
1,000+ Orient Onboard Optimize
100 1000
+
Month 1 Months 2 -10 Months 11-12
Cost of time
Transit time

19. Accumulate costs over time
2. Dynamic
Hypothesis
1. Problem Definition
3. Formulation
of model
4. Testing
5. Policy
Formulation
1,000+ Orient Onboard Optimize
100 1000
+
Month 1 Months 2 -10 Months 11-12
Cost of time
Total
compensation

20. Accumulate productive value of time
2. Dynamic
Hypothesis
1. Problem Definition
3. Formulation
of model
4. Testing
5. Policy
Formulation
1,000+ Orient Onboard Optimize
100
1000
+
Month 1 Months 2 -10 Months 11-12
Cost of time
Total
compensation
Productive
Value of time

21. Accumulate productive value of time
2. Dynamic
Hypothesis
1. Problem Definition
3. Formulation
of model
4. Testing
5. Policy
Formulation
1,000+ Orient Onboard Optimize
100
1000
+
Month 1 Months 2 -10 Months 11-12
Cost of time
Total
compensation
Productive Value
of time
Training &
Development
accessing
productive time
Orient Onboard Optimize
Training & Development
allocated across year 1.

22. 2. Dynamic
Hypothesis
1. Problem Definition
3.
Formulation
of model
4. Testing
5. Policy
Formulation
week one
productiv e hour s
deploy ed month 2 to 10
deploy ingwork for ce
~
growth ofc ompetenc y ov er time
total hour s compens ated
tr ansitioningto month 11
hours per per son per week
hours per per son per week
deploy ed month 1
total time compens ated
acc umulating compens ated time
c ounting time
acc umulating value
ofpr oduc tiv e time
tr ansitioningpeopletomonth 2
productiv e valueoftime
final weeks of year
fully competent
tottr ain hr mo 1
Year one tr aininghours
tottr ain hr mo 2to10
Training Hour s per per sonweek 1
Training Hour s per per son
week 2through 4
Training hour s per pers on
month 2thr ough 10
Training hour s per pers on
months 11 and12
productiv ity s lac k training
productiv ity s lac k
tottr ain hr finalweeks
productiv e hour s
tottr ain hrs week 1
total cost of training as
func tionofproductivity
total hour s compens ated
acc umulating oppty
c os toftraining
tottr ain hrs

23. 2. Dynamic
Hypothesis
1. Problem Definition
3.
Formulation
of model
4. Testing
5. Policy
Formulation
Training Hours per person week 1 40
Training Hours per person week 2 through 4 40
Training hours per person month 2 through 10 4
Training hours per person months 11 and 12 4
U Timing and Hours of training
88.0
95.0
154,045
208,000.0
52
Total hours compensated
Total producitive hours
Cost: Total training hours taken
during productive time
Enter hours in the table.
Press "run...
Press "reset..." to clear all values.
Year one total training hours
per person
Week
Runthe
simulation
Reset all
values
Close
application

24. • New Personnel become more
productive over time and this can
be modeled in three phases:
– Orienting new hires
– Onboarding recent hires
– Optimizing the value of experienced
hires
• Assumes there is SLACK in the
time of the person who is not
productive and training and other
development is using the slack
• Training delivered in the OPTIMIZE
phase should be high value and
enhance productivity.
• Training delivered to meet
organizational requirements is not
a best use of optimize phase
2. Dynamic
Hypothesis
1. Problem Definition
3.
Formulation
of model
4. Testing
5. Policy
Formulation
Findings

25. Policy Implications
• Implications for policy: Engaging
stakeholders
– Test assumptions.
– Adjust cause and effect relationships to
improve outcomes
– Quickly simulate multiple scenarios to
test new assumptions
– An experiential way to share the logic.
• Change the practice
– Front load training
– Take control of completion of required
training
– Re-think the timing of requirements
2. Dynamic
Hypothesis
1. Problem Definition
3.
Formulation
of model
4. Testing
5. Policy
Formulation