Slide of the Coopera(c)tion Course at Politecnico di Milano

1. Piero Fraternali
Politecnico di Milano, Italy
piero.fraternali@polimi.it

2. The evolution of the net

3. The mobile web
How many of us use a mobile terminal?

4. The social web

5. What do people do online?

6. Cosa succede in un minuto su
internet

7. How can we make good use
of all this?
Maybe helping computers help
people

8. Problems computers cannot solve

9. When computers were human
Gaspard De Prony, 1794,
hires hires hairdressers
(unemployed after
French revolution; knew
only addition and
subtraction) to create
logarithmic and
trigonometric tables. He
managed the process by
splitting the work into
very detailed workflows.
(Hairdressers better than
mathematicians in
arithmetic!)

10. Human computation
Normally the user queries the computers
What about the computer querying the user?
...or a "crowd "of users?

11. Early example: CAPTCHA
• Stands for “Completely Automated Public
Turing test to tell Computers and Humans
Apart”
• Luis von Ahn et al. coined the term in 2000
• A Program that can tell
whether a user is a human
or a computer
• Humans and machines
have complementary
skills

12. The disciplines of HC

13. Forms of HC: crowdsourcing
• Crowdsourcing is a distributed model that
assigns tasks traditionally undertaken by
employees or contractors to an undefined
crowd
– Split the task into micro-tasks
– Assign them to performers in the crowd
– Collect partial results into the final one

14. Paid Crowdsourcing: Amazon Mechanical Turk

16. Forms of HC: GWAPS
• Games with a Purpose (GWAPs)
– Exploiting the billions of hours that people spend
online playing with computer games to solve complex
problems that involve human intelligence
[vA06,LvA09].
– Useful tasks are embedded in a playful experience
where human judgment is exploited consciously or
unconsciously

17. Types of Games
[Luis von Ahn and Laura Dabbish, CACM 2008]
Three generic game structures
• Output agreement:
– Type same output
• Input agreement:
– Decide if having same input
• Inversion problem:
– P1 generates output from input
– P2 looks at P1-output and guesses P1-input

18. Output Agreement: ESP Game
• Players look at common input
• Need to agree on output

19. Input Agreement: TagATune
• Sometimes difficult to type identical output
(e.g., “describe this song”)
• Show same or different input, let users
describe, ask players if they have same
input

20. Inversion Problem: Peekaboom
•
•
•
•
Non-symmetric players
Input: Image with word
Player 1 slowly reveals pic
Player 2 tries to guess word

21. Linguistic games

22. Life science games
• Combinatorial problems with
intractable solutions spaces, in
which humans can help the
heuristic core in pruning
– Protein folding: Proteins fold
from long chains into small
balls, each in a very specific
shape
– Shape is the lower-energy
setting, which the most stable
– Fold shape is very important
to understand interactions with
out molecules
– Extremely expensive
computationally! (too many
degrees of freedom)
• A Mason-Pfizer monkey virus
retroviral protease was
modeled by FoldIT gamers in
just three weeks

23. Forms of HC: social mobilization
• Social Mobilization
– Problems with time constraints, where the
efficiency of task spreading and of solution
finding is essential
– An example of the problem and of the
techniques employed to face it is the Darpa
Network Challenge [PRP+10]
– The solution comes from the
nature of the reward
mechanism and social
ties of humans

24. HC & public resource management
• Objectives
–
–
–
–
Collect and validate data
Extract information from data
Involve people in resource usage planning and management
Change people’s behavior
• Approaches
– Passive: mine information from existing user’s activity traces
– Active: engage people in ad hoc tasks
• Ultimate goals
– Obtain “better data” for predictive models, planning and
management tool: more accurate, at finer time/space resolution,
in real time …
– Take “better decisions”: more participative, less conflicting,
capable of promoting social change

25. Extracting: population dynamics
from twitter data
• Problem: obtaining impact of population on territory at high temporal
resolution
• Can be used to detect events, estimate water consumption bursts,
waste production, etc
• Solution: using low cost geo-localized data sources (e.g., tweets)
together with structured and high cost sources (e.g., mobile phone
traces)
http://www.streamreasoning.org/demos/london2012

26. Predicting: snow fall with Flickr
images
• Problem: predicting the incidence of natural
phenomena using user generated content
• Solution: using Flickr photos tagged with “snow”
to estimate snow fall (precision 100% with 7
snow photos)
– H Zhang, M Korayem, DJ Crandall, G LeBuhn: Mining
photo-sharing websites to study ecological
phenomena. WWW 2012

27. Involving: social deliberation tools
for participatory planning
• Problem: letting a large
crowd of citizens propose
solutions or deliberate on
proposals about public
goods
• Solution: large scale
deliberation and idea
management tools
– IdeaScale.com,
MIT’s Deliberatorium
…

28. Geographical Information Systems
(GIS) + Crowd
• GeoWeb 2.0: the tools,
infrastructures and services
for the management of GIS
over the Web
• Volunteered Geo
Information (VGI): the
vision of humans as
sensors that voluntarily
create, assemble and
disseminate Web geo data.
• Participatory GIS (PGIS):
points to the social role of
GIS to promote the goals of
NGOs and groups,
especially in developing
countries
• Public PGIS: the practice of
exploiting GIS to support
public participation into
decision-making processes,
especially in developed
countries

29. Citizens' territory managment

30. Monitoring waterways: CreekWatch
• Problem: obtain simple yet useful parameters on water shed
conditions in a vast territory at low cost
• Solution: geo localized mobile+Web application
– Developed at IBM Research Almaden, 4000+ users, 25 countries
– The city of San Jose, CA, uses it to prioritize pollution cleanup efforts
• Collected data are found to have good quality

31. Other community-based PGIS
http://harassmap.org/en/
Participatory zoning project
by the Ahorani
people, Tiputini Ecuador

32. Urban games
• "Urban gaming" or
"Street Games" are
typically multi-player
location-based games
played out on city
streets and built up
urban environments
[Wikipedia
•
•
•
•
•
•
•
•
•
Find points of interest
Social analysis
Security analysis
Healthcare
Traffic analysis
Environment analysis
Collect hidden stories
Behavioral analysis
…

33. U-GWAPS Examples
• Critical City Upload
– Encourage people to make new
journeys, use public transportation
in a different way, explore new
areas
– Extra bonus points for missions,
team work, discover mysteries, etc.
• Ecopath
– track user's locations of green
activities (riding a bike or recycling
trash), connecting these sites to
define "paths" of sustainability.
– users compete with friends over
territory defined by their paths, thus
adding a social gaming context to
their actions as they make
environmentally responsible choices

34. Open problems
• Humans, like machines, can make errors
– Cognitive bias, fatigue
• Unlike machines humans can cheat
– Classification of attacks
– Spammer detection
• Quality of output improvement techniques are in
use
• Voting schemes
• Workers quality modeling and vote weighing (requires ground
truth or machine learning models and iterative / selective
labeling of data)
• Micro-flows, worker’s pre-task testing
• Task to worker assignment, active learning

35. Example of ongoing projects
Politecnico di Milano

36. CUbRIK Project
• FP7 Integrating Project
• Goals:
– Advance the architecture
of multimedia search
– Exploit the human
contribution in problem
solving & social
innovation
– Promote open-source
components
– Start up a search
business ecosystem
• http://www.cubrikproject.eu
36

37. Click-workers + expert crowd for
digital humanities research

38. GWAP for image object extraction

39. PeakWatch
• Exploiting User
Generated Content
for Mountain Peak
Detection
• Peaks automatically
identified from Flickr
photos and digital
terrain model
• Snow and water
availability prediction
46° 0’ 48.51” N
7° 48’ 6.62” E

40. PoliCrowd: a Web 3D
PGIS
– Born to promote tourism and cultural heritage
– Interaction with user mobile devices for uploading
Points Of interest (POIs)
– POIs three-dimensional visualization on World
Wind virtual globe
– User collaborative contribution in POIs
characterization
– Creating, saving and sharing customized maps
with the community
– Winner at the NASA
World Wind EU Challenge
2013

41. Step 1: find a POI & take a
picture of it

42. Step 2: report the details

43. Step 3: view your report in 2d

44. .. and in 3d

45. Step 4: enrich with multimedia
material

46. Step 6: add layers and share your
projects

47. The CrowdSearcher crowd
engagement framework
• Post tasks to the
crowd of your
choice
• Gather & analyze
results

48. Human task design:
Tips on workplaces from friends

49. Human task execution with
Facebook, Twitter, Doodle ..

50. Urbanopoly
• Buy venues, earn money and collect information about your
city by playing with the neighborhood around you in
Urbanopoly.
• Venues are real places selected from OpenStreetMap: shops,
restaurants, monuments, etc.; visit them and discover if they
are free. If the venue is free and you have enough money,
you can buy it; otherwise, if the venue is owned by another
player, you spin the wheel to see what's your destiny.

51. Web sites
•
•
•
•
•
•
•
http://www.iapad.org/multimedia.htm
www.osgeo.org
http://www.centerforcommunitymapping.org/
http://geomobile.como.polimi.it/policrowd
http://www.cefriel.com/urbanopoly
http://www.cubrikproject.eu
http://www.sketchness.com

52. References
• Managing Crowdsourced Human Computation, Panos
Ipeirotis, New York University Praveen
Paritosh, Google
• [LvA09] Edith Law and Luis von Ahn. Input-agreement:
a new mechanism for collecting data using human
computation games. In Proc. CHI 2009, 2009.
• [vA06] Luis von Ahn. Games with a purpose.
Computer, 39:92{94, 2006.
• [vAMM+08] Luis von Ahn, Ben Maurer, Colin
McMillen, David Abraham, and Manuel Blum.
recaptcha: Human-based character recognition via
web security measures.
Science, 321(5895):1465~1468, 2008.[

53. References
• Galen Pickard, Iyad Rahwan, Wei Pan, Manuel Cebrian, Riley
Crane, Anmol Madan, and Alex Pentland. Time critical social
mobilization: The darpa network challenge winning strategy. CoRR,
abs/1008.3172, 2010.
• Trant J., Exploring the potential for social tagging and folksonomy in
art museums: proof of concept. New Rev. Hypermed. Multimed.
12(1), 83–105
• Firas Khatib et al, Crystal structure of a monomeric retroviral
protease solved by protein folding game players, NATURE, 2011
• S. Kim, C. Robson, T. Zimmerman, J. Pierce, and E. M. Haber.
Creek watch: pairing usefulness and usability for successful citizen
science. In Proceedings of the 29th Int Conf on Human Factors in
Computing Systems, pages 2125–2134, New York, NY, 2011.