1.
Solving NASA’s Problems by Harnessing
the Power of the Crowd
NASA’s Center of Excellence for Collaborative Innovation
NASA Tournament Lab
Steve Rader steven.n.rader@nasa.gov 713.447.7867
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.

2.
NASA’s Center of Excellence for Collaborative Innovation (CoECI)
2
CoECI works across NASA and other U.S. Federal
Government Agencies to assist organizations in
understanding and using open innovation tools.

3.
Significant Technical Challenges to Space Exploration
• Affordable Propulsion
• Zero-boiloff Cryogenic Storage, In-space
Refueling, Improved Solar, Safe/High
Capacity/Light Energy Storage
• Long Term Human Habitation with
Limited Logistics
• High Reliability, Closed Loop ECLS, Food
Storage & Production
• Communications
• Bandwidth For Multi-spectral Imaging, Delay
• Galactic Cosmic Radiation Protection 3

4.
THE WORLD HAS CHANGED
Much of what worked for us in the past
will not work in the future!
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.

5.
90% of All Scientists that Ever Lived
are Alive Today
As 2017:
3.12 million
patents
7
The World has Changed Significantly over the Past 20 Years

6.
Processors/GPUs
CRISPR
Additive
Manufacturing
Materials/Nano-Coatings
Robotics Components
Sensors
Machine Learning Components
Open Software Components
Autonomous Vehicles
Complex Manufacturing
Cloud/Quantum Computing
Blockchain
Technology Building Blocks Accelerating the Rate of Change
Photo Credit: Betatype
The accelerating rate of change is driven by low cost
access to powerful technology building blocks and tools. 8

7.
Depth/Complexity
Breadth of Domain Expertise and Technologies
Domain-Discipline-Industry
The Breadth and Depth of Skills, Expertise, and Technology
Used by permission: Crowd Resources Consulting

8.
Depth/Complexity
Breadth of Domain Expertise and Technologies
The Breadth and Depth of Skills, Expertise, and Technology
Used by permission: Crowd Resources Consulting
Domain-Discipline-Industry
Machine
Learning

9.
Depth/Complexity
Breadth of Domain Expertise and Technologies
The Breadth and Depth of Skills, Expertise, and Technology
Used by permission: Crowd Resources Consulting
Domain-Discipline-Industry
Machine
Learning

10.
PRO
Amazing new
technologies that could
result in significant gains
towards solving hard
problems CON
Finding these
technologies & solutions
across the growing
number of possible
sources is hard
12

11.
Hard to Find Skills and
Expertise
The Rate of Change for Knowledge and
Technology is Increasing
High Risk to Remaining Competitive/Relevant
Organizations MUST Innovate!
Hard to Keep Up with
Tech Advances

12.
Source: CB Insights

13.
Average Lifespan of S&P 500 Listed Companies in Years

14.
Improving Pipeline Bundle Inspection
Subsea7 was seeking to improve pipeline inspections.
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.
16
NineSigma Challenge Result: Found a technology a fraction of the size
that performs inspections over 100x faster that will be much less expensive
to operate.
This new technology was poised to transform the industry.

15.
Depth/Complexity
Breadth of Domain Expertise and Technologies
The Breadth and Depth of Skills, Expertise, and Technology
Used by permission: Crowd Resources Consulting
Domain-Discipline-Industry
Traditional
Organization
Hiring of Skills
and Expertise

16.
Depth/Complexity
Breadth of Domain Expertise and Technologies
The Breadth and Depth of Skills, Expertise, and Technology
Used by permission: Crowd Resources Consulting
Domain-Discipline-Industry

17.
Depth/Complexity
Breadth of Domain Expertise and Technologies
The Breadth and Depth of Skills, Expertise, and Technology
Used by permission: Crowd Resources Consulting
Domain-Discipline-Industry
Individual

18.
Depth/Complexity
Breadth of Domain Expertise and Technologies
The Breadth and Depth of Skills, Expertise, and Technology
Used by permission: Crowd Resources Consulting
Domain-Discipline-Industry
Team

19.
Depth/Complexity
Breadth of Domain Expertise and Technologies
The Breadth and Depth of Skills, Expertise, and Technology
Used by permission: Crowd Resources Consulting
Domain-Discipline-Industry
The Crowd

20.
Alf Bingham Based on excerpts from “Range’ by David Epstein
PHOTOGRAPH BY BLOOMBERG VIA GETTY IMAGES

21.
“Open Innovation” is an Effective Tool to Keep
Pace with Accelerating Change
Open Innovation or crowdsourcing platforms
are bringing together and curating large
numbers of people from all over the world with
all types of backgrounds, skills, and expertise to
provide valuable products and services.

22.
Crowd Platforms are Providing Low Friction
Matching at Scale
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.

23.
Curated Communities
Freelancers
Software Coders
Film-Makers
Problem Solvers
Problem Solvers
Engineers & Designers
200,000
5,000,000
63,000,000
1,600,000
100,000
11,000,000
25
Crowd
Facing:
Company
draws in
members
from around
the world
around a
passion
and/or
interest.
Customer
Facing:
Company
mobilizes the
skills and
expertise of
their curated
community
members to
provide value
to customers.
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.

24.
26
Texas A&M
“Kyle Field” Stadium
Capacity: 102,733
Understanding Crowd Community Size
Visualizing 100,000 People

25.
Visualizing NASA’s
Workforce of 60,000
10,000
10,000
10,000
10,000 10,000
10,000

26.
Visualizing NASA’s
Workforce of 60,000
NASA
Civil Servants
17,328
1093 Scientists 7%
3000 Scientists 7%
NASA
Contractors
Est. 42,700
34,000 Total Engineers
(includes 3,200 Total
SW Engineers)
4,000 Total Scientists
24,300 Engineers
57%
(2,100 SW Engineers 5%)
6,278 Admin/
Technicians
36%
9,725 Engineers
57%
(811 Computer/SW 5%)
15,400 Admin/
Technicians
36%
28

27.
InnoCentive
400,000
Technical
Problem
Solvers
Community is 7X
larger than
NASA’s Entire
Workforce
4 Stadiums
29

28.
31
Software &
Algorithm
Developers
15 Stadiums
500,000 Focused on
Data Science/Algorithms
Community is
500X larger than
NASA’s SW Eng.
Workforce
Community is
25X larger than
NASA’s Entire
Workforce
Topcoder
1.5M

29.
550 Stadiums
0.005%=2750
Freelancer
55M
Multi-Discipline
Freelancer Workers
Community is 900X
larger than NASA’s
Entire Workforce
Equivalent to 33% of US
workforce

30.
“Most of the bright people don’t work for
you – no matter who you are.”
- Bill Joy, Co-Founder of Sun Microsystems
People
Level of Expertise/Skills
General
Population
Our
Organization
Actual Relative Size
High Value
Expertise/Skills
Crowds Provide Access to High Value Expertise/Skills
33
Curated
Community

31.
Broad participation can bring a valuable idea, missed by the experts
Credit: Rinat
Sergeev of
Laboratory for
Innovation
Sciences at
Harvard (LISH)

32.
Depth/Complexity
Breadth of Domain Expertise and Technologies
The Breadth and Depth of Skills, Expertise, and Technology
Domain-Discipline-Industry
• *Jeppesen, Lars Bo and Karim R. Lakhani. Forthcoming. Marginality and problem solving effectiveness
in broadcast search. Organization Science 20. Published Version http://orgsci.journal.informs.
Study data of successful Innocentive
challenge solutions showed…
70% of successful challenge
solutions are solved by
individuals outside of the
challenge’s specific
technical domain.
“75% of successful
solvers already
knew the solution
to the problem.”
Dr. Karim Lakhani *

33.
36
Diversity is Key to Innovation

34.
Depth/Complexity
Breadth of Domain Expertise and Technologies
Using Crowds to Solve Problems
Used by permission: Crowd Resources Consulting
Domain-Discipline-Industry
But HOW Can You Find the
“Right” Expert or the Existing
Solution from Some Other
Domain Using the Crowd?

35.
38
Accessing Crowd VALUE Using Challenges
Formulate the
Problem Statement
Design the
Challenge
Execute the
Challenge
Pick the Winner(s)
Evaluating
Get Your Solution
IP licensing and/or transfer
Solution Filtering (optional)
Well designed
challenges posed to
a curated community have
proven very effective for
discovering new and
existing (but unknown)
solutions and technologies.
A well formulated problem statement
(with good success criteria)
Solution filtering
mechanisms are
A well designed challenge
(including setting the right prize amount)
offered by
some
platforms

36.
Remove grease
from potato chips
39
The Problem

37.
Existing Solution
Mechanically vibrate the chips to shake off
the cooking oil.
Existing Solution
Mechanically vibrate the chips to shake off
the cooking oil.

38.
Remove a viscous fluid
from a delicate wafer
Remove grease
from potato chips
41
The Problem - Generalized For a Broader Crowd

39.
Winning Solution:
Acoustically vibrate the air around the chip
at resonant frequencies that cause the
cooking oil to separate from the chip.
Winning Solution:
Acoustically vibrate the air around the chip
at resonant frequencies that cause the
cooking oil to separate from the chip.

40.
Unexpected Solution Source:
A Violinist (Adjacent Domain Expert)

41.
An Case for Augmenting the Traditional R&D Approach
15 years of
proprietary
research
replicated in a
single 60-day
public
challenge
Case Study: Julian Birkinshaw, MLabnotes,
University of London Business School
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.
44
Results: Existing technology
found to solve the problem
LovePik.com

42.
Case Study
CODERS SUBMITTED SOLUTIONS DIFFERENT APPROACHES TO
SOLVE PROBLEM IDENTIFIED
WINNING COUNTRIES
RUSSIA, FRANCE, EGYPT, BELGIUM & US
122 654 89 5
Coders Submitted Solutions Different Approaches
Identified
Winning Countries
US, Russia, France, Egypt, Belgium
Improve on NIH MegaBlast algorithm
for nucleotide sequence alignment
Winning solution performs 120x faster
ANTIBODY SEQUENCE ANNOTATION
The Challenge
Improve on NIH MegaBlast algorithm
for nucleotide sequence alignment
Source:
45
47 min.
$120K
1 year
Development
4.3 hours
$2M+
Multi-year
Development
16 sec.
$6K Prize
14 Day
Challenge
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.

43.
NASA’s Center of Excellence for Collaborative Innovation
HQ
ARC
GRC
GSFC
KSC
LaRC
MSFC
JPL
JSC
SSC
AFRC
Across NASA Centers & Programs Public Facing - Worldwide Across US Federal Agencies
Supports NASA’s Space Technology Mission Directorate (STMD)
Prizes and Challenges Program
Provides services across NASA & other Federal Agencies
Virtual Office hosted within Johnson Space Center Director’s
Business Development and Technical Integration Office
Provides Access to Open Innovation (OI) Platforms Worldwide
Contracts/Mechanisms
Processes/Support
Education/Outreach
Research
27 Organizations across
15 Agencies To Date
46

44.
CoECI has both learned from and contributed to over 80 organization innovation programs
across the globe and is widely recognized as a leader in Open Innovation.
CoECI Collaboration
Center of Excellence
for Collaborative
Innovation
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.
47

45.
NASA Open Innovation Portfolio
Center of Excellence
for Collaborative
Innovation
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.
NOIS2 is our
$175,000,000 Multi-
Vendor Contract
with 32 Crowds
Access to 50+
Open Innovation
Firms &
Communities
Representing over
200 Million People
Worldwide

46.
NASA’s Internal Crowd
49

47.
Determining Urine Volume in Microgravity
51
Results: Found a working prototype at NASA
saving $1.3M & 3-5 years of development.
NASA needed improved solutions for
measuring urine volume in microgravity.

48.
Innovation & Problem Solving
Challenges
Using Challenges with Diverse Communities to develop unique and
innovative approaches to unsolved problems
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.

49.
53
Mars Balance Mass Challenge
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.
Results: A design that changed the ballast material to barium
such that when ejected during entry, they could be used as
tracers for atmospheric analysis by Mars orbiters.
NASA sought a scientific use for the “dead weight” ballast that
Mars spacecraft carried for attitude adjustments during entry.

50.
Results -
Space Poop Challenge
Results
HeroX Challenge Results: Discovered an
approach that used laparoscopic surgical
mechanisms for manipulation and passing
material across a pressure differential. Also
discovered zero-power approach to air
circulation based on medical self-inflating
ventilator technology. 5000 Submissions Total Cost: $58,000
54
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.
Winning Solutions
NASA needed to find
new and innovative
concepts to address
a significant Orion
contingency
problem with LEA
suits.

51.
Searching for Technologies
55
Using Crowd-based
Networks to Find
Technologies Invisible to
Most Internet Searches
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.

52.
Lunar Surface
Innovation Initiative
Technology Searches:
• Lunar Dust Mitigation
• 12 promising technologies found
of 82 identified
• Lunar Surface Manufacturing
• 6 promising technologies found
of 69 identified
• Acceptable Materials for the
Lunar Environment
• 8 promising materials found of
72 identified
• Lunar Dust Filters
• 1 promising technology found of
53 identified
“The technology search was rapid and comprehensive. It was also global,
which increased the odds of finding the right type of technology we were
looking for. It is very possible their findings will meet requirements for a lunar
surface experiment! I’m so happy this tech search capability exists.”
-- MSFCJennifer Edmundson
Cost Per Search: $30,000

53.
Data Science/Algorithm
Challenges
Using Challenges with Expert Communities to
Develop/Improve Algorithms.
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.

54.
Data Driven Forecasting of Solar Flares
58
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.
Results: 8-hour prediction algorithm that borrowed
techniques from RF signal processing to extract
signal from noise.
Solver was a semi-retired RF engineer (with an
unused undergraduate degree in heliophysics).
NASA needed to increase solar flare prediction from 2 hours to 4 hours.

55.
Asteroid Data Hunter
59
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.
Results: Downloadable app and algorithm
that provides a 15% Improvement in asteroid
detection over current method.
NASA needed to improve asteroid detection.

56.
60
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.
Software Challenges
Using Challenges with Expert Communities to
Develop Software Solutions.

57.
Using Competitions for Software Development
Conceptualization
Specification
Wireframes
Storyboards
Architecture
Assembly
Component Dev
Bug Hunt
Bug Race
Idea Generation
Image Credit: Wikipedia, Systems development life-cycle, http://en.wikipedia.org/wiki/Systems_development_life-cycle (as of Mar. 27, 2013, 05:48 GMT).
Source:
Each contest taps into the best of the portion of the
community that has expertise for that phase.
Concepts
Specs
UX/UI
ConOps
Arch.
Coding
Find/Fix Bugs
Creative Ideas
Projects are broken
into contests for
each phase.
Software Development

58.
ISS Astronaut Food Intake Tracker (FIT) iPad App
*Total Dev Cost w/ Add’l flight rqmts: $225,000
Results: iPad App
currently in use by ISS
crew with significant
improvement in the
capture of crew
nutrition data.
Total Cost*: $99,000
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.
JSC’s Nutritional
Biochemistry Lab needed a
better way to collect crew
nutrition data on ISS.
They used a NASA@WORK
challenge followed by a
Topcoder challenge to find
a solution.

59.
64
Creative Design
Challenges or Freelance Projects for Graphics,
Video, Animation, Writing, Digital Design/CAD

60.
8 NASA projects now
identify their project
using Freelancer
contest produced
graphics
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.
NASA Project Graphic Contests
SEE-THROUGH VERSION
COLOR VERSION GREYSCALE VERSION
WHITE VERSION
LOGISTICS
REDUCTION
LOGISTICS
REDUCTION

61.
NASA Regolith Advanced Surface Systems Operations
Robot (RASSOR) Bucket Drum Design Challenge
Common Restraint and
Mobility Aid System for
Multiple Gravity
Environments
Challenge
Advanced Lightweight Lunar Gantry
for Operations (ALLGO) Challenge
Astrobee Robotic
Arm Architecture
Challenge Series

62.
Astronaut Smartwatch App
Results: A well
designed, fully
operational
working
prototype
smartwatch app
with a data
emulator.
Total Cost:
$4,600
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.
Brian Schwing in JSC’s
Avionics Division &
Elena Naids in Mission
Ops wanted to explore
a smartwatch app to
help the ISS crew more
easily access data
during their workday.
They used a Freelancer.com to
run a User Interface design
challenge and then a software
development task all using a P-
Card.

63.
RFID-Enabled
Autonomous
Logistics
Management
(REALM)
Video Contests
$1500 storyboard
challenge followed by a
$3000 video production
challenge developed a
3 minute fully CGI
explanation of REALM.
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.
https://youtu.be/0bcWA-HnSSY

64.
Ideas
Conceptual
Designs
System
Designs
Demos/
Prototypes
Engineering
Models
Software
Algorithms/
Data Science
Graphics Video
Crowd Prog
Engagement
Technology
Search
Technology
Catalyst
103 120 17 20 30 64 54 54 25 59 108 16
NASA COECI’s
Open Innovation (OI) Products and Services
Products and Services Provided by:
Crowd-Based Prize Competitions/Challenges | Freelance Experts/Workers | Micro-Tasking | OI Program Support
69
Technical Solutions Multimedia
700+
Projects
94% Were
Successful
77% Have Cost
Savings
48% Average
Cost Savings
$79M+ Estimated
Cost Savings

65.
“The Future of Work”
We are in the midst of
a significant shift in
how work is
performed and how
organizations will get
work done.
Source The Human Cloud, the Gig Economy, & the Transformation of Work
Report by Staffing Industry Analysts (www.staffingindustry.com) 2017

66.
Crowd Platforms are Providing Low Friction
Matching at Scale
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.

67.
Source: Freelancing in America 2017
Edelman Intelligence – commissioned by Upwork and Freelancers Union
Growth in Freelancer Workforce is 3X the Overall Workforce
"41.5% of the average organization’s total
workforce is comprised of non-employee
labor, a figure that is nearly double what it
was roughly seven years ago"
- Christopher Dwyer, VP of Research at Ardent Partners, 2019

68.
Source: Freelancing in America 2017
Edelman Intelligence – commissioned by Upwork and Freelancers Union
If Trends Continue, Freelancers will Become the U.S. Workforce
Majority in Less than a Decade
The COVID-19
Pandemic has
accelerated
this trend.

69.
2/3/23
1/2 Say They Would Not Take a Traditional Job
No Matter How Much They Were Offered

70.
Lifelong Learning is Now Mandatory to Keep Up

71.
Freelance/Gig/Crowd workers model for lifelong learning
far surpasses training offered in large organizations

72.
COVID-19 is Pushing Companies

73.
Time Needed to Start Building New Skills
Online in Jobs of Tomorrow

74.
The Emerging Open Workforce
On-Demand and Persistent
Available both Globally and Locally
Grunt Workers and Experienced Experts
Lifelong Learners Up to Speed on the Latest Tech
Workforce Platforms Providing Low Friction Access
Source: Public NASA presentation

75.
Trade names, trademarks, and logos are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.
84
Access to Freelance
Freelance Experts

76.
Adapting to the Future of Work
Open platforms are
bypassing the gatekeepers
Everyone has an
opportunity to participate
Organizations need
flexible workforces
that can adapt to
changing needs
and skills
Workers will increasingly have
the flexibility and agency to
find work/life balance and
pursue their passions through
lifelong learning
Image by Karolina Grabowska from Pixabay

77.
86
Diversity is Key to Innovation

78.
Image by G Poulsen from Pixabay
Perspectives on Change

79.
Crowds, gig-workers, freelancers are a rapidly growing
resource with increasing capabilities
Curated communities are attracting passion and
building expertise and skills
Open methods are extremely effective for
accessing valuable innovations
Those that fail to innovate will be left behind
“OPEN” is the Future and
“INNOVATION” is No Longer Optional

80.
Significant Technical Challenges to NASA’s Mission
89
If we are serious
about leading in
Space Exploration,
we must use new
tools if we hope to
access the
necessary
technologies, ideas,
and innovations we
require.

81.
Discussion & Collaboration
http://nasa.gov/solve steven.n.rader@nasa.gov
90

82.
@SteveRader @NASASolve

83.
Background Reading

84.
The challenge owner was the CIA who needed to find a
way of tracking Russian actors in Crimea but needed to
keep the actual problem secret.
Methods for Protecting Sensitive Challenges:
Contextual Obscuration
A challenge was posted to find a way to track a specific bison in
Yellowstone National Park using only images posted on social media.
An innovative
solution was found
and deployed
without detection.

85.
The challenge owner was the Navy who wanted to
find a way to capture the energy from the powerful
electrical arc created when it fired its rail gun.
Methods for Protecting Sensitive Challenges:
Technical Obscuration
Challenge: If you are in an enclosed metal space
surrounded by water and there are lighting strikes, is
there any way to capture and use the power?