Through its flexible model of knowledge and learning services, NASA meets the development needs of practitioners, project teams, and the organization. By linking business strategy to knowledge and learning approaches, NASA provides an integrated and systematic approach to address critical skillsets for technical, leadership, and business capabilities. This approach optimizes individual competence, project team performance, and organizational learning in a way that enables NASA to meet the changing needs of its workforce. SPEAKER BIOGRAPHY Dr. Jon Boyle has served in several capacities in public and private sector organizations, from industrial production lines and overseas military combat units to multinational corporations, NASA flight facilities, and academia. He possesses expertise in Cognitive Neurosciences, Industrial/Organizational Psychology, Knowledge Management, Group Processes, Human Resources and Workforce Development, Business Strategy, Technology-Enabled Learning, Research and Development, and Process Improvement. Jon currently serves as the NASA Agency Deputy Chief Knowledge Officer (InuTeq), where he contributes to the development of the overall NASA Technical Workforce through Knowledge Services. He earned a B.A. in Psychology and Biology from the University of Southern Maine; a M.Ed. from Boston University; a M.A in Industrial and Organizational Psychology from George Mason University; and a Ph.D. in Human Development from Virginia Tech, as well as participating in diverse training and certifications in technology, project management, quality-related topics, acquisition and procurement, leadership, and coaching. He currently teaches several undergraduate and graduate programs and maintains an active research and publication agenda. Jon lives in the DC Metro area with his wife Allyson, son Zachary, and twin daughters Bevin and Riley. His son Christopher recently returned from Afghanistan where he serves as a Blackhawk Crew Chief in the U.S. Army and is now stationed at Fort Belvoir, VA.

1. Close Encounters of 3 Kinds
Attracting, Retaining and Leveraging Talent to
Support an Organizational Knowledge System at
NASA
Dr. Jon Boyle
NASA Chief Knowledge Office

3. The Library of Babel – Jorge Luis Borges

4. I. Knowledge at NASA
- Generational Knowledge
- The Changing Landscape
- Products, Projects, Entrepreneurship
- Complexity
- Stakeholder Messages

5. Knowledge Spans Generations
X-15 Space Shuttle
One of the X-15’s many knowledge legacies that it
passed to the Shuttle was unpowered landing —
both reentered the atmosphere as gliders.
Introduced: 1958 Retired: 2010

6. The Changing Knowledge Landscape
• Managing knowledge is nothing
new at NASA.
• Many early efforts were in
response to specific needs.
• In recent years, agency
stakeholders have identified
opportunities for greater
coordination and collaboration
across NASA.

7. Projects, Products, Entrepreneurship
Complex Project-
Based Organization
Mass-Production
Organization
Entrepreneurial
Organization
Product One-and-only Scalable manufacture Permanent beta
Problems Novel Routine Hackable
Technology New/invented Improved/more efficient Frugal
Cost Life cycle Unit -> Zero marginal
Schedule Project completion Productivity rate Iterative
Customer Involved at inception Involved at point of sale Involved in testing
Knowledge
Need
Innovation Continuous improvement Bootstrap + innovation

8. Confusing, vague, and poorly defined priorities, strategies,
lines of authority, governance, policies, roles, responsibilities,
support
Multiple customers,
stakeholders, and
partners at multiple
levels of interest,
involvement,
responsibility
Technical complexity and
system integration issues
within & across multiple
disciplines and systems
Increasing
amounts of data
and information
for process input,
throughput,
output
Multiple overlapping, conflicting,
outdated processes and
procedures involving multiple
POCs across multiple levels &
across multiple oversight &
advisory entities
COMPLEXITY
Complexity at NASA

9. Message from Stakeholders 2002-2012 (1)
GAO 2002: “…fundamental weaknesses in the collection
and sharing of lessons learned agency-wide.”
ASAP 2011: “…recommends NASA establish a single focal
point (a Chief Knowledge Officer) within the Agency to
develop the policy and requirements necessary to
integrate knowledge capture…”
OIG 2012: “…inconsistent policy direction and
implementation for the Agency’s overall lessons learned
program.”

10. Message from Stakeholders 2002 – 2012 (2)
PMC Quarterly
Knowledge Reviews
Chaired FKMC and conducted Federal Benchmarking
Bi-annual rotating CKO
community strategic meetings
Chief Knowledge
Officer appointed (OCE)
Knowledge Management
Policy Authorized
Designed & deployed
NASA Knowledge Map
Established NASA Knowledge
web portal KM.NASA.GOV
Agency CKO integrated
community
Initiated Publishing & Communication
of NASA Knowledge
CKO Search, Find, Visualization Working Group
Critical Knowledge Study
Critical Knowledge Digital Tools Gateway Activity
Knowledge Referee Process
approved by OCE & Deputy Administrator
Benchmark Boeing KM Portal,
PMI Global Executive Council,
Merck, FKMC
2014
2015
2013
Knowledge 2020 Critical Knowledge
Gateway Benchmarking

11. II. Areas of Progress
- Policy and Governance
- Management Imperatives
- Knowledge Community and Networks
- Knowledge Services Strategy
- CKO Roles and Responsibilities
- The 4 As
 Knowledge Transfer (Chris Scolese)
 Career Development Framework
 Technical Skills (B. Gerstenmaier)
- Knowledge Map and km.nasa.gov

12. Policy and Governance
NASA collaboratively developed and adopted a new
knowledge policy in November 2013. Key features:
- Federated approach to governance.
- CKOs appointed at Centers, Mission Directorates,
Functional Offices, with Roles and Responsibilities.
- Tools such as the first NASA Knowledge Map based on 6
activity categories that form a common vocabulary and
km.nasa.gov to focus communications and distribution.

13. NASA Management Requirements
• Supports and extends Knowledge Services gains for
the NASA Technical Workforce towards improved
accessibility, searchability, findability, and
visualization.
• No additional cost.
• Least administrative burden.
• Formal, rigorous, iterative, and Senior Leader
supported.
• Integrated, reinforcing, and actionable.
• Measurable and objective.

14. NASA Knowledge Community and
Networks
• Federal KM
Working
Group
• APQC
• PMI
• Int’l
• Columbia
• Agency CKO
• Local CKOs/POCs
• Communities of
practice
NASA
Government Industry/
Professional
Organizations/
Academia

15. Knowledge Services Strategy
Enable
accessibility,
findability,
searchability,
and visualization
of
data,
information and
systems.
Facilitate
opportunities
through better
communications
and processes
for sharing and
networking.
Establish best
practices for
capturing &
retaining,
sharing &
applying,
discovering &
creating
knowledge.
Establish
maturity model
for knowledge
effectiveness to
measure and
validate.
Respect local customs & enhance organizational norms
(The Federated Approach).
The goal: Where does the NASA Technical Workforce go to find and use the
critical knowledge required now and in the future to achieve mission success
in a highly complex and unforgiving environment?”

16. CKO Role and Responsibilities (1)
Given the complex nature of knowledge at NASA,
the agency has adopted a Federated model for
coordination of knowledge activities.
The NASA CKO functions as a facilitator and
champion for knowledge.

17. CKO Roles and Responsibilities (2)
Autonomy
Each Center and Mission
Directorate determines
the approach that best
meets its needs.
Responsibility
Knowledge applicable to
all NASA missions and
Centers will be shared to
the extent possible across
the entire Agency.
The Federated Model
+

18. Individual Responsibility: 4 A’s

19. Organizational Responsibility:
Transferring Knowledge

20. Ability - Career Development Framework
ENTRY
PROJECT TEAM MEMBER OR TECHNICAL ENGINEER
MID-CAREER
SMALL PROJECT MANAGER OR SUBSYSTEM LEAD
MID-CAREER
LARGE PM OR SYSTEMS MANAGER
EXECUTIVE LEVEL
PROGRAM OR VERY LARGE PROJECT MANAGER
Core: Foundations of Aerospace at NASA
Obtain mentor
Join professional associations
Core: Project Management & Systems Engineering
In-depth courses; team lead assignments; Project HOPE
Attendance at technical conferences or knowledge sharing activities
Core: Advanced Project Management & Systems Engineering
Mentoring
In-depth courses; rotational assignments
Participation in knowledge sharing activities
Core: Executive Program
Mentoring; Administrator’s Executive Forum
Leadership by example in knowledge sharing
Non-traditional and hands-on
learning experiences
Developmental assignments
APPEL core curriculum
Cohort selected by
NASA senior leaders
Performance enhancement
for teams
Knowledge sharing forums
Knowledge sharing forums
Performance enhancement
for teams
LEARNING STRATEGIES

21. ““...it's still hard to give up the
technical side. I am a recovering
engineer. But I recognize you just
can't do that stuff anymore and
to think you still have those skills
is also really wrong…“
- Bill Gerstenmaier, HEOMD Associate Administrator
career stages

22. Knowledge Map (1)
• Online resource at km.nasa.gov
• Information hyperlinked and sortable by:
–Organizations
–CKOs/points of contact
–Knowledge categories (see next slide)

23. Knowledge Map (2)

25. km.nasa.gov
Links, resources, and updates
CKO communications

26. III. The Road Ahead
- Strategic Knowledge Imperatives
- REAL Knowledge KS Model
- Process Gaps
- Big Challenges
- Critical Knowledge Referee Process
- Digital Tools
- Close/ Questions

27. Strategic Knowledge Imperatives (1)

28. Strategic Knowledge Imperatives (2)
• Leadership: Without leadership, KS results are at best
serendipitous, at worst fail.
• It is a Project World: An adaptable discipline that maximizes use
of learning to promote efficiency and effectiveness.
• Knowledge: Organized set of content, skills, and capabilities
gained through experience and formal and informal learning that
is applied to make sense of new and existing data and
information.
• Talent Management: Specification, identification, nurturing,
transfer, maintenance, and expansion of the competitive
advantage of practitioner expertise and competence.
• Portfolio Management: Integrates projects with strategy and
creates an organizing framework and focus driving organizational
purpose and activities.
• Certification: Objective, validated standards and functions to
benchmark achievement in defined categories of practitioner
performance and capability.

29. Strategic Knowledge Imperatives (3)
• Transparency: Nothing hidden for long, especially errors.
• Frugal Innovation: Viewing constraints as opportunities in an
era of restricted and diminished resources.
• Accelerated Learning: Broadest view of learning using digital
technologies, knowledge-sharing, learning strategies, social
media, cross-discipline content.
• Problem-centric Approach: Non-partisan, non-biased, non-
judgmental, pragmatic orientation to problems and solutions,
focusing on achievement, improvement, and innovation.
• Governance, Business Management and Operations: Pragmatic
alignment, oversight, approvals, and implementation of project
operations that are not administratively burdensome.
• Digital Technology: Can result in open, social network-centric,
non-proprietary, adaptable, and flexible frameworks to
accelerate learning.

31. NASA’s Gaps in Core Knowledge Processes
Capture Share
Discover
Mature capability:
Case studies
Lessons Learned Info. System
Videos
Shuttle Knowledge Console
Knowledge-based risk records
Mature capability:
Online tools and portals
Face-to-face events
Communities of practice
Inadequate capability:
Search – enhanced ability to discover
Culture – expectation to discover
“Nudges” – reminders to discover

32. Big Challenges
• Findability, Searchability, Adaptability
• Prioritization of Agency Critical Knowledge
• What are the metrics and measures that capture
effectiveness and efficiency in the core
knowledge processes?
• What is the relationship between Knowledge
Services, accelerated learning, and reducing
complexity?
• Can an understanding of biases and heuristics
that drive organizational and societal
expectations help organizations make better
decisions and design better knowledge services?

33. Example: Agency Critical Knowledge Referee Activity (1)

34. Example: Agency Critical Knowledge Referee Activity (2)
1.0 PEOPLE 2.0 PROCESS
3.0 KNOWLEDGE TRANSFER & DIGITAL TECHNOLOGY 4.0 DISCIPLINE TECHNICAL
1.1 Raise issues that impact mission
success & performance.
1.2 Failure in development is ok
as long as people learn from it.
1.3 Stay curious about better
processes and improvement.
1.4 Think outside of community of practice &
consider strategic implications.
1.6 Engineers hesitant to accept risk
because they do not make decision to take
on these risks.
1.5 NASA employs more technical
authorities than technical workers.
1. 1.7 Lessons of failure are forgotten
during relaxation period.
2.1 Innovate for affordability, good versus perfect,
see how specs/standards were derived.
2.2 Encourage technicians and engineers to consider
innovation and cost in addition to technical excellence.
2.3 Be mindful of affordability and
repeatability like successful Shuttle model.
2.4 Make risk-informed decisions.
2.5 Be wary of requirements put on ourselves.
2.6 More emphasis on understanding “why” behind a
process, not blindly following the precise standard.
3.9 Lessons must happen at the Engineering
Directorates & the Center level.
29. 2.7 The Portfolio level is where norms are
set & must be communicated from.
14. 3.7 Capture knowledge at critical program milestones &
transfer knowledge to new program/ project formulation. 13. 3.8 Integrate NASA lessons into on-boarding activities.
17. 3.4 Identify useful ways to integrate lessons
learned.
16. 3.5 Include a succinct abstract with knowledge content. 15. 3.6 Share a periodic lesson at beginning of
standing meetings.
21. 3.1 Would like to use activities like SMD Principal Investigator
(PI) Forum & put online.
19. 3.2 Workforce needs ability to find information in a
meaningful way.
3.3 Knowledge best gained when people actually experience
a discipline, meet people & work real issues.
18.
1.8 Really vital that all people raise
honest concerns and problems early.
38.
1.9 MAVEN’s success largely due to effective
communications & team building.
31.
1.10 Value the importance of project reviews
using experienced people for sharing critical
knowledge
1.11 Infuse new project managers
with gray-hair reviews.
36.
2.8 A consistent formal process to communicate mission problems is vital.
2.9 Risk management critical for identifying
& communicating critical knowledge. 2.10 Many risks pertain to personnel who need to be watched and coached.
2.11 Each problem needs to be addressed,
communicated, worked openly and swiftly.
3.12 Hold practitioners accountable to specific best practices.
3.
3.13 Ensure consistent regular documentation of
lessons learned.
9.3.14 Should do case study for unique missions.
2.12 Critical knowledge results from effective program & project management.
27.
3.15 PI Forums are valuable for SMD university-led missions.
8.
4.2 Requirements grow at beginning, then need to manage limited
budgets, so project loses engineering rigor that adds to cost.
4.5 The engineering/project team must
be comfortable and skilled in balancing
capability (performance), cost,
schedule and risk.
2. 4.3 Prepare engineers to effectively make trades &
balances between requirements and cost.
4.4 The requirement may not be attainable within
cost, but the capability may have more flexibility.
1.12 Must have culture of
communication.
4.1 If there is no recent failure or mishap, can focus away
from sharing, learning, testing and conversation in order to
do things faster or cheaper.
1.13 Workforce must be free to speak
up & say what is on their mind.
35.
28. 1.14 Eliminate toxic management.
1.15 Workforce must know strategy & promote
behaviors that support direction.
37.
1.16 Must understand how to change a
situation when a team is struggling.
34.
1.17 How to facilitate in a virtual
environment.
1.18 Employees must know
our critical missions.
33. 1.19 Place accountability for
the culture on the people
30. 1.20 Tear down silos & stovepipes.
2.13 Establish top five list of the most critical
knowledge & communicate across NASA.
3.9 Create videos of expertise.
3.10 Link critical knowledge to messages & communications.
3.11 Create social collaboration & media location
to support culture.
11.
1.21 Need to remind people that
failure can happen to you.
32.
3.16 Shouldn’t fail because of lesson learned
that should have been learned earlier.
7.
2.16 A waiver must have a rational justification
based on solid engineering work & data.
25.
1.22 Poisonous managers or technical experts
who shut down communications are bad.
2.14 When budget is tight, wrong approach is to
save by reducing solid engineering discipline.
26. 2.15 For success, it is necessary to be
preoccupied with failure.
6. 3.17 Executive talks are very useful.
24. 2.17 Managers need to be careful about relying
too much on an “inner circle” of advisors.
23. 2.18 A manager must be open & accept they
do not have all the technical expertise.
5. 3.18 Case studies at NSC are important
lessons applicable for NASA.
4. 3.19 Program & project rules based on lessons learned is a great practice.
10. 2.19 Senior leadership should ensure their expectations
and critical lessons flow down to branches.
4.6 Decision-making biases (Normalization
of Deviance, Group Think, Abilene
Paradox) are very dangerous.
4.7 Make sure technical expertise and learning
is being maintained & updated.
1.
1.23 Teams need to talk to each other to generate the
lessons learned.
3.20 Forum approach with NASA leaders
talking about Aero, technologies, objectives,
Safety and Mission Assurance & Center
expertise was extremely powerful as it
provided tremendous context.
2.21 Most problems are in Formulation Phase.
2.20 Decision-making is a key area for learning.
3.21 Project teams need to take formal time to discuss real lessons learned.
3.22 Offer project teams &
leaders tools (lessons learned
reviews, storytelling, process
mapping).
3.23 Lessons learned should be shared across the broader workforce.
4.8 Aero Research and Development is not the same as
operations and development.
4.9 Distributed Roughness Element on the GIII at
Dryden went poorly; today there are more cooperative
agreements to better track performance as opposed to
grants.
4.10 High Ice Water Content project had decision and
implementation problems, but the lessons learned were
done extremely well.
1.24 Need to communicate more of a mindset of
knowledge to be innovative.
3.24 Encourage leaders & managers to communicate critical lessons to workforce.
1.25 Early career initiatives & partners to promote hands-on learning & innovation,
including young professionals
2.22 Strike balance on need for disciplined technical rigor & using agile approach.
2.23 Focus on assessing the true proposal value &
actual value compared to cost.
2.25 Have annual program reviews establishing
program expectations.
2.24 Determining appropriate cost reserves for missions.
3.25 Use formal lessons learned capture approach.
2.26 How to be agile & innovative with inconsistent funding profiles.
2.27 Should better use Center innovation funds.
4.11 Better structure partnerships
targeting non-traditional players.
Critical Knowledge Referee
Disposition Activity
· Training/Development
· Knowledge Service
· HCM
· Policy & Procedure
· Process
· Awareness
· Other
Legend
· HEMOD Green
· SMD Blue
· HCM Orange
· OSMA Purple
· AERO Black
· STMD Red
Category Definitions
· People (Factors involving communications,
individual behavior, team behavior,
organizational culture, attitudes,
predispositions and expectations).
· Process (Factors addressing specific
actions towards defined outcomes in a
systems perspective).
· Knowledge Transfer & Digital Technology
(Factors involving moving knowledge
across organizational boundaries and
digital information and communications
tools that enable and accelerate
interaction and learning)
· Discipline Technical (Factors involving
content and lessons related to specific
domains of practice)

35. Digital Tools for Critical Knowledge (1)
• km.nasa.gov serves as integrating mechanism
and critical knowledge gateway.
• Data and information visibility, searchability,
findability, and visualization are key factors
driving Agency improvement efforts for Data
Management, Knowledge Services, and
Analytics.
• Critical Knowledge from Knowledge Referee
Process drives priorities and administrative
actions.

36. Digital Tools for Critical Knowledge (2)
• Additional actions on modern digital tools are moving rapidly by
leveraging KM federated infrastructure. Examples:
 Creation of JPLTube video with spoken keyword search capabilities.
 Capture & sharing lessons at GSFC of over 50 Case Studies & direct
support of JPL, GRC & MSFC case development & digital distribution.
 LaRC Oral Lessons Learned documentation & digital distribution.
 KSC analysis & update of Agency Lessons Learned Information System
(LLIS) Database.
 Active analysis at JSC of cutting-edge Searchability & Findability.
Capabilities & Shuttle Console development that captures thousands of
documents & lessons related to the program.
 NASA CKO Office Agency Knowledge Map update with new content &
interactivity including a new HEOMD Knowledge-Based Risk Dashboard.
 New Masters with Masters video series on Lessons Learned and Critical
Knowledge & digital distribution.
 Young Professionals assisting NASA in citing best digital tools.
 Benchmarking best-in-class Knowledge Services (Boeing and others)

37. Digital Tools for Critical Knowledge (3) - Find a
Document
Like this…… Instead of this
Source: http://www.yasiv.com/amazon#/Search?q=graph%20drawing&category=Books&lang=US

38. Digital Tools for Critical Knowledge at NASA (4) - Find Similar
videos
Like this…… Instead of this
http://www.yasiv.com/youtube#/Search?q=project%20management%20risk

39. Digital Tools for Critical Knowledge at NASA (5) -
Search Lessons Learned
Source: http://cs.ucsb.edu/~jod/topicnets.html
Like this…… Instead of this

40. Digital Tools for Critical Knowledge at NASA (6) - Find
Experts
Source:
http://cs.ucsb.edu/~jod/papers/C-6-
LinkedVis-IUI2013.pdf
Like this…… Instead of this

42. Questions
web: km.nasa.gov