Presentation to Rice University Professional Masters of Engineering/Science Program.

1. Harnessing the
Power of the Crowd
Using Crowd-Based Challenges as
Tools for Engineering Projects
NASA’s Center of Excellence for
Collaborative Innovation (CoECI)
Steve Rader steven.n.rader@nasa.gov
@NASA_NTL

2. Food Supplies
Clean
Water
Rising Sea Levels
Global Warming
Asteroids

3. 3

4. 4

5. 5
We are used to the model of “the expert”, that keeps up with
all of the tech advancements that we need for a given field.
In the past, our biggest struggle has been to find and keep
these experts!
But now, the
expert you
need today is
different than
the expert you
need
tomorrow….
The pace of technology development is just too much.
Individuals
cannot possibly
keep up with
the technology
advancements

6. Even if you could find the right expert for your problem, could you afford them?
Would you be able to work through the red tape to get them to work with you?
What would you do with him after you have your answer and you
need to move to the next expert area?
What about all of the technology advancement that
is going on out there that doesn’t
NEED an expert…
it just needs you to find it!
It is all like looking for
a
needle
in a
haystack!

7. 7
But
we now
have an
advantage!
A global
population
that
is becoming
increasingly
CONNECTED
by the INTERNET.

8. 8
Internet User Growth from 2000-2012 and Potential
Asia
1.1B
28%
Europe
519M
63%
Africa
167M
16%
North
America
274M
79%
Latin
America
255M
43%
Middle
East
90M
40%
Oceana/A
ustrialia
24M
68%
The
World
2.4B
34%
Users in
2000
Additional
users by 2012
Population
Not on the
Internet YET
Current #
of Users
% of Population
that is Online
566% Growth
from 2000-2012
LegendBased on table from: http://www.internetworldstats.com/stats.htm

9. 9
With more
and more of
the world’s
population on
the Internet,
the ability to
access
people with
unique skills,
expertise, and
experience is
rapidly
increasing.
Academic students
researchers
faculty
Retired scientists,
engineers, &
programmers
Current
“under-employed”
scientists, engineers,
& programmers
The Tip of the Iceberg! Not only are the Billions of people around the world
rapidly getting online. Right now, the 2.8B people that ARE online are just
barely aware of these platforms and what they mean to them…

10. 10
7 Billion
Population
10
2.4 Billion People on the Internet
People on
Crowdsourcing
Platforms
Estimated
30-50
Million

11. 11
Cognitive
Surplus
This “interest” in using their skills to make a difference
or win a prize outside of normal employment (at
night or on the weekends) can be described as….
People seem genuinely interested in engaging in
problem solving and being “the one” to make a
difference!

12. 12
If two heads
are better
than one….
Now consider how this cognitive surplus begins to become a powerful
force as more of the current 2.8 billion connected individuals join in the
efforts and the remaining 4.2 billion become connected on the Internet.
The possibilities to harness the creativity of the human mind are enormous!

13. Image: Illumination Entertainment, Universal Studios
But just because 2.8 Billion people
are connected does NOT mean
that they are organized ….
AT ALL…
In fact it can be chaos.

14. 14
In fact… it’s fascinating what global
connectivity has meant to many.

15. 15
Networks & Communities
Curated Communities
Inventors
Software Coders
Film-Makers
Photographers
Graphic Artists
Engineers & Designers
800,000*
84,000*
1,081,000
745,000
50,000
1,860,000
*estimate
Well formulated crowd-based platforms actively work to
build a community of users that are passionate enthusiasts.

16. 16
Curated Communities
Resources and
Tools for
Members
Incentives for
Members to
Participate
User Agreements
for Privacy and
Payment
Mechanism for
Handling IP
Licensing and/or
Transfer
Community
Building &
Communication
Curated communities
are built around
enabling people to
pursue their passion
and create a win-win
for the company and its
community members.
They provide structure
and incentives.
Communities do NOT
like to be exploited!

17. GoldGutsGoodGlory
Why Does The Crowd Contribute?
Earn Money (real or virtual)
Have Fun (or pass the time)
Socialize with Others
Obtain Recognition or Prestige
(leaderboards, badges)
Do Good (altruism)
Learn Something New
Obtain Something Else
Create Self-Serving Resource
17
Crowdsourcing & Human Computation Labeling Data & Building Hybrid Systems
by Matthew Lease, Assistant Professor at University of Texas at Austin on May 03, 2013
http://www.slideshare.net/mattlease/crowdsourcing-human-computation-labeling-data-building-hybrid-systems
Multiple
Incentives
can often
operate in
parallel

18. Effectively Using Communities
Create an
Innovative New
Solution
Apply an
Existing
Technology
(in an innovative way)
Access Best
Possible
Product or
Service
(competition winner)
Find an
Existing
Solution
(you didn’t know existed)
Access Very
Specific
Expertise
(found through
competition)
18
Solve a
Problem
Develop a
Product
Provide a
Service
Innovation from Diversity found via Challenges
(Experience, Context/Perspective, Expertise)
Expert or Domain Focused MembershipDiverse Membership
High Quality Products/Services
(via Competition to get Best in Domain)

19. 19
The Crowd Just
Gave Me 3500
Solutions…
Now What
Do I Do?

20. 20
If you aren’t careful,
you end up sorting
through a heap of
potential solutions … …most of which ar
NOT what you need

21. 21
Crowdsourcing
innovative solutions is
really a lot more like
mining…. You need
special processes and
tools to extract the
gold from the ore.

22. 22Executing a Crowd-Sourced Challenge
Formulate the
Problem Statement
Design the
Challenge
Execute the
Challenge
Pick the Winner(s)
Judging
Get Your Solution
IP licensing and/or transfer
Solution Filtering (optional)
Knowing how
to do all of
these steps
really helps to
mitigate the issues
associated with this
“too many solutions” problem.
ALL of these steps can help to
minimize the number of solutions
you end up needing to judge.
A well formulated problem statement
(with good success criteria)
A well designed challenge
(including setting the right prize amount)
Solution filtering
mechanisms are
offered
by some
platforms

23. We Can Use the Crowd as
a Force Multiplier…
23
…To Do MORE with LESS

24. www.infographically.com
This is an established and GROWING INDUSTRY!

25. 25
Crowdsourcing is Mainstream

26. What is NASA Doing with Crowdsourcing?

27. NASA Innovation Pavilion

28. 28
“One Stop Shop” Website which Lists ALL NASA Challenges
http://www.nasa.gov/solve

29. Centennial Challenges
NTL Curated Community Challenges
Citizen Science Challenges
Space Apps Challenge
Software
Apps
Scientific
Observations/C
ontributions
Tech Dev &
Demo
Years
Months
Months
Days/We
eks
Duration Prizes Product Participation
$M+
$1K-
$100Ks
Recognition
Recognition
Ideas,
Designs,
Software
29

30. 30
First Things First!
NASA has some of
the most amazing
employees on the
PLANET!
Tap into that
crowd FIRST!

31. 31
Over 15,000 Registered Members
(25% of NASA’s 60,000 CS & Contractor Workforce)
15-20 Challenges per Year
Growth of the NASA@work
Community since October 2012
2-4 Active challenges
posted at any one time
New challenge posts every
~2-3 weeks
People that work at
NASA want to make
a difference!

32. Innovation & Problem Solving
Using Challenges with Diverse Communities to
develop unique and innovative approaches
to unsolved problems

33. NASA Innovation Pavilion on
33
Global Community
of 350,000+ Solvers
Diverse Member Base
High Success Rate in
Solving Hard Problems
with Innovative
Solutions
Map showing participation of 2900 solvers
from 80 countries in NASA’s pilot challenges

34. “Diverse perspectives and tools
enable collections of people to
find more and better solutions.”
34
Diversity Trumps Ability Theorem:
“The people you’d least expect to
solve a problem were exactly the
ones most likely to crack it.”
Scott Page
Author of Difference:
How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies

35. 35
Diversity is the Key to Innovation
One MIT study into InnoCentive
revealed that solvers were more
successful when they had less
experience in the relevant discipline.
Some data suggests that as much as
70% of successful InnoCentive
challenge solutions are solved by
individuals outside of the challenge’s
specific technical domain.

36. 36
“In 60 days, Roche was able to solve a problem that it
and its partner have been tinkering with and optimizing
for the last 15 years. The solutions provided actually
mirrored the entire history of Roche’s R&D programme.
All of the solutions Roche had tried came in. “
Swiss company with 80,000
employees, Roche operates in 150
countries and has R&D operations in
Europe, North America and Asia-
Pacific
Roche is a world leader in in vitro
diagnostics.
Julian Birkinshaw, MLabnotes, University of London Business School
Roche ran an InnoCentive challenge

37. 37
$20,000 in prizes
Over 2800 registrants
219 Entries
Winning Submission: Barium tracers for atmospheric analysis
Winner was Ted
Ground from
Rising Star, Texas
(population 799)

38. Galactic Cosmic Ray Mitigation
Finding ways to protect humans from the
effects of cosmic galactic rays in deep
space.
Non-Invasive Intra-Cranial Pressure Monitoring
Finding methods to monitor the pressure in
the human brain that are non-invasive.
Strain Measurement of Vectran and Kevlar
Webbing
Seeking improved methods for strain
measurement under very specific
conditions for inflatable habitat testing.
Additional
InnoCentive Challenges
38

39. Algorithm Competitions
Leverage Competition to Optimize Complex
Algorithmic Problems

40. Case Study
CODERS SUBMITTED SOLUTIONS DIFFERENT APPROACHES TO
SOLVE PROBLEM IDENTIFIED
WINNING COUNTRIES
RUSSIA, FRANCE, EGYPT, BELGIUM & US
122 654 89 5
Improve on NIH MegaBlast algorithm
for nucleotide sequence alignment
Winning solution performs 120x faster
ANTIBODY SEQUENCE ANNOTATION
Source:
47 min. 16 sec.4.3 hours
$2M+
Multi-year
Development

41. 15% Improvement!
Over current method of
identifying asteroids in
the main belt of Asteroids
that orbit between Mars
& Jupiter
1241 Registrants
625 Solutions Submitted
$74,124 in Prizes Awarded
Algorithm AND App

42. Asteroid Tracker
Optimized algorithm for tracking multiple
asteroids with an array of antennas.
Earth Science
Developing ideas for tools that use NASA
earth science data to help local
communities with climate resilience.
Planetary Data System (PDS) - Cassini
Image processing of Saturn rings data to
detect possible new satellites or moons.
Additional Recent Harvard/
TopCoder Algorithm Challenges
42

43. Using Competitions for Software Development
Conceptualization
Specification
Wireframes
Storyboards
Architecture
Assembly
Component Dev
Bug Hunt
Bug Race
Idea Generation
Image Credit: Wikipedia, Systems development life-cycle, http://en.wikipedia.org/wiki/Systems_development_life-cycle (as of Mar. 27, 2013, 05:48 GMT).
Source:

44. Lunar Mapping and Monitoring Project (LMMP)
Has reduced image processing times from
18 hours to 3 hours.
Disruption/Delay Tolerant Network (DTN)
Attempting to solve a problem with
distributing security keys in a disrupted/
delayed network. Building out the protocol
stack.
ISS Food Intake Tracker
An iPAD app designed for the ISS to better
track their food intake (featured at the
Apple Developers Conference in 2014).
Recent Harvard/TopCoder
Software Development Challenges
44

45. Leverage Competition to Optimize
Engineering Designs
Design Challenges

47. 47

49. GrabCAD 30 Day challenge for $3000: 492 CAD Designs Submitted

51. Using New Methods to Search for New and
Emerging Technology to Meet NASA’s Needs
Searching for Technologies

52. 52

53. Yet2.com
Provides a “matching” service that finds
technologies and solutions from industry, academia,
and/or individuals for a given need/challenge.
Includes a 130,000 member community and links to
over 16,000 commercial entities.
Very effective (and cost effective) in searching for
existing products or development efforts.

54. Yet2.com Challenges
Bone Density Measurement
Monitoring of Water and
Biocides
Non-Invasive Intra-Cranial Pressure Measurement
“Much more than we expected! Very pleasantly surprised
that this process exposed so many potential solutions with
such wide breadth and depth”
“Learned that we should have revisited technologies that
we rejected earlier
81 Leads Identified
63 Rejected
3 High Interest Solutions
5 Other Interesting Solutions
6 Potential Complementary
Technologies
2 Potential Solutions

55. Non-
129 Challenges
4 Tech Surveys
2 Videos
1 Design
14 Algorithms
15 Software
2 Ideation
2 Graphics
76 Challenges
NASA Innovation Pavilion
9 Theoretical
2 Ideation
2 Reduction to
Practice
NASA Challenge Experience
with Curated Communities
Includes NASA and other U.S. Gov’t Agency Challenges executed under NASA’s CoECI

56. Crowd-Based Challenges are valuable tools that
should be considered for future engineering
projects
Effective Method to Find Truly Innovative Solutions
Key Method to Find the Right Existing Technologies
(out of the sea of R&D work across a variety of domains)
Efficient Method to Augment Project Teams
with Specialized Skills as Needed
Using Crowds as Engineering Tools

57. Steve Rader
steven.n.rader@nasa.gov
@steverader

58. Acknowledgements
References:
Crowdsourcing Landscape: www.crowdsourcingresults.com, www.infographically.com
Global Internet Use: Based on table from: http://www.internetworldstats.com/stats.htm
Howe, Jeff (2008-08-18). Crowdsourcing: Why the Power of the Crowd Is Driving the Future
of Business, Doubleday Religious Publishing Group.
“Diverse perspectives and tools enable collections of people to find more and better
solutions.”
Scott Page. Difference: How the Power of Diversity Creates Better Groups, Firms, Schools,
and Societies.
Karim R. Lakhani et al., “The Value of Openness in Scientific Problem Solving.”
Graphics:
experts - http://directoryofexcelexperts.com/
needle in haystack - http://siliconbeachclearly.com/
global network - http://www.emirates.com/
iceberg - http://www.weirdoptics.com/iceberg-illusion/
minions - Illumination Entertainment, Universal Studios
lolcats – http://background-pictures.picphotos.net/
farmville – http://farmville.startpagina.nl
tinder – http://apptiinder.com
thumbsdown – http://commons.wikimedia.org
community – http://www.blogworld.com
minivan – http://vzyalslidingdoor.blogspot.com
milking machine - greenfusestock.photoshelter.com
rocket scientists – LIFE magazine
innovation – http://abovethecrowd.com
problem solving – http://creativerealities.com
equations – http://gallaryhip.com, http://imaginenvision.com
designs – http://www.grabcad.com