Lawrence Suda speaks about “Using Simulations to Train Future Project Leaders at NASA” at Serious Play Conference 2012 ABSTRACT: The essential message of this presentation is about training future project managers at NASA and about project team behavior: how the simulation technology behaves, how people behave and how the simulation and people behave, and sometimes misbehave, together. Ultimately the real test of the people on the project is not so much what they know; it’s what they do with their knowledge. Making a decision is not the same as implementing it. Knowledge, insight and project wisdom are needed to fully execute a successful project.

1. Using Simulations to Train Future
Project Leaders at NASA
Experience.
Lawrence Suda

2. Performance Simulations
• People learn by doing in a risk-free
environment
• Learners:
– are encouraged to experiment
– learners drive the experience
– get realistic and individualized
feedback throughout the experience
– must master certain behaviors to
achieve higher overall performance
metrics

3. Business Simulations
Project Leadership Experience
Adaptive Project Leadership
Applied Agile Project Management
Global Project Leadership
Portfolio & Program Leadership
Business Acumen
Virtual Project Leadership
Change Leadership
Operations Management
Strategic Business Management
Supply Chain Management

4. Simulations Mapped to Levels
4

5. Business Simulations Scope
Large-Scale Team Exercise – 24 to 35 +
hours to complete
Simulation Integrated into Learning
Design – Mixed Methodologies
Open-ended, Multi-layered, Multi-
dimensional
Systems and Detailed Levels of
Complexity
Instructor Facilitated but Learner Driven
Classroom or Virtual or Both
Teams Are Not Competing

6. Validation
Vetted over 30 years with world class organizations and
literature review
– NASA’s PM Model
– NASA Project Mirror: Assessment & Development Center
– GE Project Leader Research & Model
– Nortel Networks PM Model
– MITRE
– Royal Bank of Canada
– Department of Energy
– Merck Leadership Model
– Management Research Group Leadership Model
– NASA Team Study & Assessment
– NASA Leadership Model
– US Navy Project Execution Study
– PMI Career Development Framework
– Alcatel-Lucent
– Hewlett-Packard

7. Transitioning Into Project Leadership
Program/Project Manager Levels
Level 1 Level 2 Level 3 Level 4
High
Technical
Discipline Actions
Program/Project
Management Actions
Low

8. Dimensions
Program/Project Manager Levels
Level 1 Level 2 Level 3 Level 4
High
• Technical --- Leadership Technical
Discipline Actions
• Technical --- Abstract
• Productive --- Practical Program/Project
Management Actions
• Narrow --- Broad Low
• Task Orientation --- World View
• Defined Goals --- Moving Goals
• Discipline Specific --- Multidisciplinary

9. Major Clusters
Competency Research
Focus Groups & Individual Interviews
 Personal Leadership
 Team Leadership
 NASA Agency Acumen
 Communication Effectiveness
 Strategic Perspective and Thinking
 Project Management Skills and Knowledge

10. o n
A cti
d
in ate lic
ub tion
ord P c
e
Co e fl
R
eam ing
T n
L ear
t
oin ing
J n in g
n e an
Pla d M
Team Learning
are
Sh

11. Workshop Themes and Modules

12. Learning Objectives
The overall purpose of the workshop is to improve our understanding
of human and project complexities and to develop effective
approaches to leading complex projects. More specifically we want to
develop skills in:
• Improving project team performance
• Leading in complex environments
• Developing adaptive leadership skills
• Developing defensible, flexible, adaptive plans
• Using information to make better decisions
• Identifying and managing project tradeoffs
• Understanding unintended consequences of decisions
• Recognizing when to focus on technical versus adaptive
problems
•Learning from successes and failures

13. Typical Agenda
Introduction Plenary
Case Diagnosis Simulation (Teams)
Case Discussion Plenary
Theory Plenary
Planning Simulation Teams
Execution Simulation Teams
Debrief Plenary
Application Plenary

14. NASA Project Leadership Lab
Day 1 Day 2 Day 3 Day 4 Day 5
Introduction Film Film Project Review Film
Scheduling Personal Styles Team Work &
Leadership Context Knowledge Transfer After Action
Setting Reviews
360 Feedback
Team Styles
Simulation Feedback To
Team leadership Teams
Resource Planning Presentation Simulation
Mind Mapping Reflections
Simulation Simulation Action Planning
Lunch Lunch Lunch Lunch Lunch
Introduction to Risk Management Project Execution
project simulation
Definition Simulation
Scheduling
Simulation
Quality & Budget
Management Simulation
Simulation Complete
Simulation Simulation
Film

15. 4-Day Learning Design

16. Short Lectures
• Defensible Planning
• Project Execution
– Problem Solving, Decision
Making
• Team Leadership
– Performance, Motivation, Start-
up & Maintenance
• Adaptive Leadership

17. Planning Phase
Hear Lectures & Simulation Planning
 Scheduling
 Resource
 Risks
 Quality
 Budgeting
Create & Defend Plan

18. Example Tools – Scheduling & Resourcing
Time (week number)
CR CR CR CR CR CR
TR
15- 1- 8- 15- 22- 15- 1- 8- 15-
1-Jan 8-Jan Jan 22-Jan 1-Feb 8-Feb 15-Feb 22-Feb Mar Mar Mar Mar 1-Apr 8-Apr Apr 22-Apr May May May 22-May 1-Jun 8-Jun 15-Jun 22-Jun 1-Jul 8-Jul
# H
Staff Quality
T
Task ID ppl SD W PR CC W IM C
TE
1
2
3
4 5 6 7
8 9
10
11
12
13
14 15
16
17 18
19
20 21
22 23
24
25
26
In the workshop
participants develop plans
1 2 X X X 2 2 2
3 2 X X 2 2 2
6 2 X X 2 2 2 2 2
7
8
9
1
2
3
X X
X X
X
X
X
1
1
1
X
1
X
1
X
1
3
X
X
3
X
X
3
X
X
X
X
X
X
X
using both computer and
16
19
20
2
2
3
X X
X X X
X X X X X
2
2
2
2
2
2
2
X
3
X
3
X
3
manual tools
21 2 X X X 2 2
2 1 X X 1 1 1 X X X
4 1 X 1 1 X X X
5 2 X 2 2 X X X
10 2/V X X X 2 2 2 2 2 X X X
11 2/V X X X 2 2 2 2 2 X X X
12 2/V X X 2 2 X X X X X X X X
13 2 X X 2 2 2 X X X X X X X
14 2 X 2 2 X X X
15 2 X X X 2 X X X
17 4 X X 4 4 4 X X
18 2 X X 2 2 2 X X
Total
staff 2 2 2 3 3 3 8 8 9 9 9 8 7 5 10 10 10 4 4 3 3 3 2 2
H T TE
SD W PR CC W IM C Note
Senna 7 7 9 6 5 6 9 1 1 1 3 3 3
Mayes 4 4 8 0 8 6 6 1 1 1 3 3 3
Franz 2 3 7 4 8 0 4 CC CC HW 2 2 2
Banks 3 7 4 3 4 1 4
Dance 7 4 8 3 7 7 8
Bilko 6 5 7 3 6 0 6
Inc
Bovis
Brown
Samin
Unter
Ltd
Burns 6 5 6 0 7 0 6
Candy 9 6 9 5 10 8 9
Mindy 10 9 3 9 3 0 7
18

19. Planning
4-person teams (instructor shown) develop an overall strategy and approach.
They develop a detailed plan that includes: project priorities, critical path
schedule, workload leveling, human resources plan, training, risk
management, quality, budgeting. They must defend their strategy and plans.
19

20. Project Execution Phase
Hear Lectures & Simulation
Control
 Problem Solving,
 Decision Making
 Decision Trade-offs
 Re-planning
 Control Cycle
 Control Tools

21. Implementation
Teams make tactical decisions on three levels as they implement their plan:
Project, Task and Staff. During the exercise, they must actively manage the
project as conditions change as a result of their decisions and unknown events
that create more issues and problems. Note: Instructor can introduce more issues
and problems. Participants report status on a monthly basis. They have control
tools to assist them as they proceed through the simulation exercise.
21

22. Some Typical Disruptions
– Resignations & Absenteeism
– Resources pull for other projects
– Large Scope Change
– Scope Creep
– Customer wants more meetings
– Sponsor wants project sooner
– New technology doesn’t work
– Team performance is low
– Key staff member resigns
– Budget cuts
– Estimates are wrong
– Poor Morale

23. Project Competition

24. Performance Summary
KPI PLANNED ACTUAL
Final Cost 693,000 875,000
Completion Week / Date June 17 June 20
Open Quality Issues 0–2 0
Quality Issues Reported 30 – 40 47
Quality Issues Fixed 30 – 40 47
Labor Cost / Hour 64 74
Technology Cost / Hour 60 75
Overtime Costs 6,315 20,486
Training Costs 10,000 4,000
Utilization % 83 87
Performance % 100 83
Absence Rate % 2 1
Communication % 6 4
Number of Resignations 1 1
Staff Pulled Away Weeks 6-8 5

25. Simulation Debrief
• What were your results? (brief overall description)
• Did you keep your risk mitigation plan updated?
• What contributed to your successes?
• What were your major simulation / project challenges?
• How did you overcome any challenges?
• Describe your team dynamics.
• Did you capitalize on team diversity?
• Did you notice movement through the team development
process?
25

26. Reported Learning
• Lead in ambiguous, complex environments
• Improve individual performance
• Develop team effectiveness and improve team performance
• Develop defensible, flexible plans
• Align project work with organization’s strategy
• Think strategically and conceptually
• Translate strategies into concrete actions
• Plan systematically and establishes objectives and priorities
• Manage risk -- knows when too much risk is being taken, when the
right amount is being taken, know the consequence and communicate
to the team and to leadership
• Learn, develop and model new team behaviors
• Develop personal leadership skills
• Demonstrate sensitivity and encourage innovation and creativity

27. NASA Comparison of PM62, PM61, PM60, PM59
Participants’ Mean Ratings
5 4
Mean Mean 3.8
4.8
Rating Rating
4.6 3.6
4.4 3.4
4.2 3.2
4 3
3.8 2.8
3.6 2.6
3.4 2.4
3.2 2.2
3 2
1 2 3 4 5 6 7 8 9 10
PM62 PM61 PM60 PM59 PM62 PM61 PM60 PM59
Questions about Instructor and Session Questions about Information Presented in Session
1. Instructor Effectiveness 6. Relevant to job
2. Discussion were effective in reinforcing learning 7. Likeliness to apply back on the job
(not asked in PM59) 8. Usefulness in increasing knowledge/skill to be an effective
3. Activities were effective in reinforcing learning Project Manager
(not asked in PM59) 9. Value to assist in the achievement of goals as a Project
4. Instructor stimulated thinking Manager
5. Overall rating of learning experience 10. Usefulness back on the job

28. Accreditations
• PMI Global Registered Provider
Advanced PM = 21 to 35 PDU’s
• American Council on Education (ACE)
Accredited for Graduate Credits
– Stevens Institute of Technology
– Drexel University
– Duke CE
– e-Cornell
– EADA (Barcelona Spain)

29. Application
• High Potential Leadership Training
• New Employee Orientation
• Introduction of New PM Initiatives
• Kick-off or Wrap-up of Conferences
• Create Project-based Organization
• Culture Change
• PM Certification Capstone

30. Current & Future Challenges
 Manager Involvement - Getting Participants’ Managers to be part of the
learning process.
 Training Time Compression - Pressures to reduce training time and time away
from work.
 Work Pressures on Participants – Work while at training
 Travel Funds – Budgets are reduced.
 Budget Pressures – Develop funds for new work reduced.
 Training Outsourced to One or Few Vendors.
 Measuring Effectiveness – ROI.
 Technology Challenges & Learning Design Challenges – Particularly with
Large Scale Sims in Virtual Environments
 Changing Organization, Program, Project Dynamics – e.g. Agile PM

31. Thank You!
Lawrence Suda
Palatine Group, Inc.
New York City, NY
Lsuda@thepalatinegroup.com

32. Useful Links:
www.seriousplayconference.com
www.seriousgamesdirectory.com
www.seriousgamesassociation.com
Contact:
sbohle@seriousgamesassociation.com