Crowd Science: Measurements, Models, and Methods - Presented at Hawaii International Conference for System Sciences, 2016

1. Crowd Science:
Measurement, Models, and Methods
John Prpić & Prashant Shukla
HICSS 2016

2.  Overview of the Field
 Research Goals
 Theoretical Grounding
 Toward Crowd Science
 Discussion
The Grand aim of science is to cover the greatest number of experimental facts by
logical deduction from the smallest number of hypotheses or actions.
- Albert Einstein.
Agenda
2

3.  Crowdsourcing - Crowdflower, Wikipedia, TopCoder,
 Citizen Science - GalaxyZoo, Foldit, Zooniverse,
 Crowdfunding - Kickstarter, Indiegogo, Kiva,
 Open Innovation Platforms - Kaggle, Innocentive, Challenge.gov,
 Sharing Economy - AirBnB, Uber, Lyft,
 Public Sector - IPaidaBribe, FixMyStreet, Open Ministry,
IT-Mediated Crowds – Practice
3

4.  Human Computation - (Ipeirotis, Michelucci, von Ahn)
 Open Collaboration - (Benkler, Majchrzak, Mako-Hill)
 Ideas Competitions - (Afuah, Boudreau, Lakhani)
 Citizen Science - (Cooper, Crowston, Meier)
 Crowdfunding - (Agrawal, Burtch, Mollick)
 Public Sector - (Aitamurto, Brabham, Noveck)
IT-Mediated Crowds – Research
4

5.  We’re observing increased research and practice on
organizations using IT to connect with dispersed individuals for
explicit resource creation purposes.
 This state of affairs precipitates the need to precisely measure
the processes and benefits of these activities over myriad
different implementations.
Research Motivation
5

6.  We seek to address these salient and non-trivial considerations
by laying a foundation of:
 Theory,
 Measures,
 Research methods,
 That allow us to test Crowd-engagement efficacy across
organizations, industries, technologies, and geographies.
Research Goals
6

7. The Theory of Crowd Capital (Prpić & Shukla 2013; 2014)
Dispersed Knowledge (Hayek 1945)
Every individual has private knowledge that is useful, but cannot be accessed.
Crowd Capability
The IT structure, form of content, and internal processes through which an organization engages a Crowd.
Crowd Capital
A heterogeneous organizational resource generated from IT-mediated Crowds.
Theoretical Grounding
7
Dispersed
Knowledge
Crowd
Capability
Crowd Capital

8. IT Structure
Crowd-engaging IT is found in Episodic or Collaborative forms, distinguished by
whether the individuals in a Crowd interact with one another or not, through the IT
(Prpić & Shukla 2013; 2014).
Theoretical Grounding
8

9. Theoretical Grounding
9
Typology of Crowd-derived content & Organizational processing
methods (Prpić, Shukla, Kietzmann & McCarthy 2015)

10. Theoretical Grounding
10
Comparison of Common Characteristics of Crowdsourcing Techniques
(Prpić, Taeihagh & Melton 2015)

11.  An empirical apparatus that considers counterfactuals in ascertaining
the benefits of various implementations of IT-mediated Crowds.
 Currently, to test hypotheses about the benefits of using IT-mediated
Crowds, researchers use data from a single Crowdsourcing,
Crowdfunding, Open innovation platform.
 Need to consider counterfactuals.
 Can’t quantify the benefits of using Crowds otherwise.
 Can’t generalize, can’t predict.
 Can’t move toward a science of IT-mediated Crowds.
Toward Crowd Science
11

12. Counterfactuals
 If the use of IT-mediated Crowds is the treatment, we need to measure the
difference between the treatment and control group, before and after
implementing a Crowd.
Toward Crowd Science:
Counterfactuals
12

13. Operationalization
 Measures of processes and benefits across a variety of IT-mediated Crowds
implemented.
Toward Crowd Science:
Operationalizations
13

14. Experiments
 Randomly select organizations/units seeking specific and similar resources from
IT-mediated Crowds.
 Observe how they do with respect to Crowd Capital generation relative to the
control group over a period of time.
Toward Crowd Science: Methods
14

15. Meta-Analysis
 Focus on quantitative meta analysis that accumulates the evidence from the
extant hypothesis testing endeavors in the field.
 Revolves around collection of effect sizes/coefficients and applications of
procedures such as meta analytic regression analysis (MARA) and
homogeneity analysis (HOMA).
Toward Crowd Science: Methods
15

16. Natural Experiments
 Difference-in-differences techniques.
Toward Crowd Science: Methods
16

17.  Unprecedented shocks to knowledge production function
 Unprecedented on-demand scale of human participation.
 Unprecedented on-demand speed and aggregation of human effort.
 Unprecedented on-demand access to human knowledge.
 New outcomes & new configurations of socio-technical systems.
Why we need Crowd Science
17
The Grand aim of science is to cover the greatest number
of experimental facts by logical deduction from the
smallest number of hypotheses or actions.
- Albert Einstein.

18.  Are we moving toward more perfect information through Crowd
Science?
 Can Crowd Science optimize stewardship of common-pool resources?
 Crowd vs. Market vs. Firm?
 AI, IoT, Machine Learning, with Crowds?
Crowd Science: Open Questions
18

19. Twitter:
 @JPnuggets
 @Prashshukla
Blogs:
 phdinstrategicmanagement.wordpress.com
 creativecommodum.com
Thank You!
19