Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Meet the Crowd (final)
1. Meet the Crowd:
The Political Economies and Cultural
Meaning of Digital Labor
Syed Shoaib Ali, Microsoft Research India
Deepti Desai, Microsoft Research India
Mary L. Gray, Microsoft Research New England
Sara Kingsley, Microsoft Research NYC
Kate Miltner, Microsoft Research New England
Gregory Minton, Microsoft Research New England
Rajesh Patel, Microsoft Bing Redmond
Siddharth Suri, Microsoft Research NYC
http://research.microsoft.com/crowdwork
4. India Crowd Household Income
*AMT Survey Respondents
21% of Survey Respondents living
in India report receiving financial
assistance*
$193.92/year (USD)
=
All India Poverty Line
(Government of India, Planning
Commission, 2013)
< $450 $450 -
$1,000
$1,000 -
$1,500
$1,500 -
$2,000
$2,000 -
$2,500
$2,500 -
$3,000
$3,000 -
$3,500
$3,500 -
$4,000
$4,000 -
$4,500
> $4,500
5. USA Crowd Household Income
U.S. Median
Household
Income
=
$51,371
(U.S. Census, 2012)
At least 14% of Survey
Respondent Households live
under the Federal Poverty Line
(U.S. Department of Health and Human Services Poverty Guidelines, 2013)
$31, 589
=
Poverty Line
6 person household
(U.S. Department of Health and Human
Services Poverty Guidelines, 2013)
AMT Survey Respondents
6. Crowd Working Hours (AMT)
U.S. Survey Respondents
29 Hours per Week
=
Average Hours of Work,
US Employees Working
from Home
(U.S. Bureau of Labor Statistics, 2013)
India Survey Respondents
AMT Survey Respondents
< 1 1-5 6-10 11-20 21-30 31-40 40 + < 1 1-5 6-10 11-20 21-30 31-40 40 +
7. Task Specialization (AMT)
AMT Survey Respondents
AMT workers who do one type
of task (specialize) mostly do
surveys
4
=
Average Number of Task
(HIT) Types AMT Workers
do
8. Worker Variety (UHRS)
Total work time per day over a 4 week period for 4 sample workers on UHRS. Different
colors represent different tasks (HITapps).
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
Su Tu Th Sa M W F Su Tu Th Sa M W F
Hoursofworkcompleted
0
1
2
3
4
5
6
7
8
Su Tu Th Sa M W F Su Tu Th Sa M W F
Hoursofworkcompleted
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Su Tu Th Sa M W F Su Tu Th Sa M W F
Hoursofworkcompleted
0
0.5
1
1.5
2
2.5
Su Tu Th Sa M W F Su Tu Th Sa M W F
Hoursofworkcompleted
Consistent, Single Task Worker Consistent, Varied Tasks Worker
Sporadic Worker Transient Worker
9. Time Variation (UHRS)
Variation in work volume over 4 weeks
S
u
M T
u
W T
h
F S
a
S
u
M T
u
W T
h
F S
a
S
u
M T
u
W T
h
F S
a
S
u
M T
u
W T
h
F S
a
VOLUMEOF
WORK
VOLUMEOFWORK
Time (EDT)
Variation in work volume over 1 day
10. HIT Discovery
Based on 4856 responses to a task (HIT) posted on AMT, April 23 – May 28, 2014
0
50
100
150
200
250
300
350
400
450
W Th F Sa Su M Tu W Th F Sa Su M Tu W Th F Sa Su M Tu W Th F Sa Su M Tu W Th F Sa Su M Tu
Online forum General search Referred by someone Searching for this requester Other No response
11. Mapping the Crowd
Self-reported locations for about 10,000 participants in a map task (HIT) on AMT.
Coloration of counties/districts is by population density.
12. Mapping the Crowd
Self-reported locations for about 10,000 participants in a map task (HIT) on AMT.
Coloration of counties/districts is by an estimate of Internet access.
13. Interview Locations
Locations of the 48 interviews
conducted to date.
Coloration is by estimated density of workers on AMT.
This estimate is based on our map task (HIT).
Editor's Notes
Even though most of our survey participants *likely* fall above the “All India Poverty Line” – 21% reports that they receive financial assistance of some sort, suggesting that their current earned household income is not enough to meet their needs.
Follow up research question: what technological impediments might prevent crowdworkers from doing certain tasks like image tagging? If crowdworkers in India cannot afford better quality Internet, they may avoid tasks that they believe will cause delays in HIT start and submit times, especially since delays in downloading HITs could negatively affect reputation.
Citations:
[1] U.S. population data comes from the U.S. Census Bureau, at the Department of Commerce. Available online, at:
[2] Indian population data comes from the Indian Census. Available online, at:
Citations:
[1] Internet access data comes from the OOKLA net index database. Data available online for download, at: http://www.netindex.com/
