SlideShare a Scribd company logo

Community-based Crowdsourcing

Slides presented at the 2nd International Workshop on the Theory and Practice of Social Machines (at WWW2014 conference in Seoul)

1 of 16
Download to read offline
Dipartimento di
Elettronica, Informazione e
Bioingegneria
COMMUNITY-BASED
CROWDSOURCING
Marco Brambilla, Stefano Ceri, Andrea Mauri, Riccardo Volonterio
Dipartimento di Elettronica,
Informazione e Bioingegneria
Crowd-based Applications
 Emerging crowd-based applications:
– opinion mining
– localized information gathering
– marketing campaigns
– expert response gathering
 General structure:
– the requestor poses some questions
– a wide set of responders are in charge of providing answers
(typically unknown to the requestor)
– the system organizes a response collection campaign
SOCM’14, Monday, April 7
Community-basedCrowdsourcing
3
Dipartimento di Elettronica,
Informazione e Bioingegneria
Our approach
 Combines a conceptual framework, a specification
paradigm and a reactive execution control environment
 Supports designing, deploying, and monitoring
applications on top of crowd-based systems
– Design is top-down, platform-independent
– Deployment turns declarative specifications into platform-
specific implementations which include social networks and
crowdsourcing platforms
– Monitoring provides reactive control, which guarantees
applications’ adaptation and interoperability
SOCM’14, Monday, April 7
Community-basedCrowdsourcing
4
Dipartimento di Elettronica,
Informazione e Bioingegneria
Control
 Controlling crowdsourcing tasks is a fundamental issue
– Cost
– Time
– Quality
 In our approach
– Implemented trough active rules
• Control is easily expressible
 Simple control data structures
 Familiar formalism
• Support to arbitrarily complex controls
 Extensibility mechanisms
• Active rules can be system-generated
 Well-defined semantic
SOCM’14, Monday, April 7
Community-basedCrowdsourcing
5
Dipartimento di Elettronica,
Informazione e Bioingegneria
Community
Set of people that share
 Interests
 Feature
..or belong to
 common entity
 social network
SOCM’14, Monday, April 7
Community-basedCrowdsourcing
6
Dipartimento di Elettronica,
Informazione e Bioingegneria
Leveraging communities
 Why?
– Experts
– More engaged
 How?
– Determine the communities of performers
– Monitor them taking into the account the behavior of their
member
• Community Control
SOCM’14, Monday, April 7
Community-basedCrowdsourcing
7

Recommended

Answering Search Queries with CrowdSearcher: a crowdsourcing and social netwo...
Answering Search Queries with CrowdSearcher: a crowdsourcing and social netwo...Answering Search Queries with CrowdSearcher: a crowdsourcing and social netwo...
Answering Search Queries with CrowdSearcher: a crowdsourcing and social netwo...Marco Brambilla
 
Choosing the right crowd. Expert finding in social networks. edbt 2013
Choosing the right crowd. Expert finding in social networks. edbt 2013Choosing the right crowd. Expert finding in social networks. edbt 2013
Choosing the right crowd. Expert finding in social networks. edbt 2013Marco Brambilla
 
CrowdTruth @VU Faculty Colloquium (June 2015)
CrowdTruth @VU Faculty Colloquium (June 2015)CrowdTruth @VU Faculty Colloquium (June 2015)
CrowdTruth @VU Faculty Colloquium (June 2015)Lora Aroyo
 
Social Web 2014: Final Presentations (Part II)
Social Web 2014: Final Presentations (Part II)Social Web 2014: Final Presentations (Part II)
Social Web 2014: Final Presentations (Part II)Lora Aroyo
 
Lecture 7: How to STUDY the Social Web? (2014)
Lecture 7: How to STUDY the Social Web? (2014)Lecture 7: How to STUDY the Social Web? (2014)
Lecture 7: How to STUDY the Social Web? (2014)Lora Aroyo
 
User Behaviour Pattern Recognition On Twitter Social Network
User Behaviour Pattern Recognition On Twitter Social NetworkUser Behaviour Pattern Recognition On Twitter Social Network
User Behaviour Pattern Recognition On Twitter Social NetworkGeorge Konstantakopoulos
 
How information spreads on social networks when unexpected events occur
How information spreads on social networks when unexpected events occurHow information spreads on social networks when unexpected events occur
How information spreads on social networks when unexpected events occurFarida Vis
 

More Related Content

What's hot

Human Computation for Big Data
Human Computation for Big DataHuman Computation for Big Data
Human Computation for Big DataeXascale Infolab
 
Lecture 5: Personalization on the Social Web (2014)
Lecture 5: Personalization on the Social Web (2014)Lecture 5: Personalization on the Social Web (2014)
Lecture 5: Personalization on the Social Web (2014)Lora Aroyo
 
Lecture 4: How can we MINE, ANALYSE & VISUALISE the Social Web? (2014)
Lecture 4: How can we MINE, ANALYSE & VISUALISE the Social Web? (2014)Lecture 4: How can we MINE, ANALYSE & VISUALISE the Social Web? (2014)
Lecture 4: How can we MINE, ANALYSE & VISUALISE the Social Web? (2014)Lora Aroyo
 
Social Computing in the area of Big Data at the Know-Center Austria's leading...
Social Computing in the area of Big Data at the Know-Center Austria's leading...Social Computing in the area of Big Data at the Know-Center Austria's leading...
Social Computing in the area of Big Data at the Know-Center Austria's leading...Christoph Trattner
 
User-centered research for developing programs & articulating value.
User-centered research for developing programs & articulating value.User-centered research for developing programs & articulating value.
User-centered research for developing programs & articulating value.Lynn Connaway
 
Mining the Social Web - Lecture 1 - T61.6020 lecture-01-slides
Mining the Social Web - Lecture 1 - T61.6020 lecture-01-slidesMining the Social Web - Lecture 1 - T61.6020 lecture-01-slides
Mining the Social Web - Lecture 1 - T61.6020 lecture-01-slidesMichael Mathioudakis
 
Zook making sense of geosocial media-final
Zook making sense of geosocial media-finalZook making sense of geosocial media-final
Zook making sense of geosocial media-finaloiisdp
 
Hashtag Conversations,Eventgraphs, and User Ego Neighborhoods: Extracting So...
Hashtag Conversations,Eventgraphs, and User Ego Neighborhoods:  Extracting So...Hashtag Conversations,Eventgraphs, and User Ego Neighborhoods:  Extracting So...
Hashtag Conversations,Eventgraphs, and User Ego Neighborhoods: Extracting So...Shalin Hai-Jew
 
Damage Assessment from Social Media Imagery Data During Disasters
Damage Assessment from Social Media Imagery Data During DisastersDamage Assessment from Social Media Imagery Data During Disasters
Damage Assessment from Social Media Imagery Data During DisastersMuhammad Imran
 
Studying Online Food Consumption and Production Patterns: Recent Trends and C...
Studying Online Food Consumption and Production Patterns: Recent Trends and C...Studying Online Food Consumption and Production Patterns: Recent Trends and C...
Studying Online Food Consumption and Production Patterns: Recent Trends and C...Christoph Trattner
 
Ethical and Legal Issues in Computational Social Science - Lecture 7 in Intro...
Ethical and Legal Issues in Computational Social Science - Lecture 7 in Intro...Ethical and Legal Issues in Computational Social Science - Lecture 7 in Intro...
Ethical and Legal Issues in Computational Social Science - Lecture 7 in Intro...Lauri Eloranta
 
Image4Act: Online Social Media Image Processing for Disaster Response
Image4Act: Online Social Media Image Processing for Disaster ResponseImage4Act: Online Social Media Image Processing for Disaster Response
Image4Act: Online Social Media Image Processing for Disaster ResponseMuhammad Imran
 
Finding media illustrating events
Finding media illustrating eventsFinding media illustrating events
Finding media illustrating eventsRaphael Troncy
 
How to Analyze the Privacy of 750000 Smartphone Apps
How to Analyze the Privacy of 750000 Smartphone Apps How to Analyze the Privacy of 750000 Smartphone Apps
How to Analyze the Privacy of 750000 Smartphone Apps Jason Hong
 
A comparison of Lexicon-based approaches for Sentiment Analysis of microblog ...
A comparison of Lexicon-based approaches for Sentiment Analysis of microblog ...A comparison of Lexicon-based approaches for Sentiment Analysis of microblog ...
A comparison of Lexicon-based approaches for Sentiment Analysis of microblog ...Cataldo Musto
 
CHI2015 - Citizen Science || Zooniverse
CHI2015 - Citizen Science || ZooniverseCHI2015 - Citizen Science || Zooniverse
CHI2015 - Citizen Science || ZooniverseRamine Tinati
 

What's hot (20)

Human Computation for Big Data
Human Computation for Big DataHuman Computation for Big Data
Human Computation for Big Data
 
Lecture 5: Personalization on the Social Web (2014)
Lecture 5: Personalization on the Social Web (2014)Lecture 5: Personalization on the Social Web (2014)
Lecture 5: Personalization on the Social Web (2014)
 
Lecture 4: How can we MINE, ANALYSE & VISUALISE the Social Web? (2014)
Lecture 4: How can we MINE, ANALYSE & VISUALISE the Social Web? (2014)Lecture 4: How can we MINE, ANALYSE & VISUALISE the Social Web? (2014)
Lecture 4: How can we MINE, ANALYSE & VISUALISE the Social Web? (2014)
 
Rogers digitalmethods 4nov2010
Rogers digitalmethods 4nov2010Rogers digitalmethods 4nov2010
Rogers digitalmethods 4nov2010
 
CSE509 Lecture 5
CSE509 Lecture 5CSE509 Lecture 5
CSE509 Lecture 5
 
Social Computing in the area of Big Data at the Know-Center Austria's leading...
Social Computing in the area of Big Data at the Know-Center Austria's leading...Social Computing in the area of Big Data at the Know-Center Austria's leading...
Social Computing in the area of Big Data at the Know-Center Austria's leading...
 
Clarkson - Joshua White - Research Proposal Presentation
Clarkson - Joshua White - Research Proposal PresentationClarkson - Joshua White - Research Proposal Presentation
Clarkson - Joshua White - Research Proposal Presentation
 
User-centered research for developing programs & articulating value.
User-centered research for developing programs & articulating value.User-centered research for developing programs & articulating value.
User-centered research for developing programs & articulating value.
 
Mining the Social Web - Lecture 1 - T61.6020 lecture-01-slides
Mining the Social Web - Lecture 1 - T61.6020 lecture-01-slidesMining the Social Web - Lecture 1 - T61.6020 lecture-01-slides
Mining the Social Web - Lecture 1 - T61.6020 lecture-01-slides
 
Zook making sense of geosocial media-final
Zook making sense of geosocial media-finalZook making sense of geosocial media-final
Zook making sense of geosocial media-final
 
Hashtag Conversations,Eventgraphs, and User Ego Neighborhoods: Extracting So...
Hashtag Conversations,Eventgraphs, and User Ego Neighborhoods:  Extracting So...Hashtag Conversations,Eventgraphs, and User Ego Neighborhoods:  Extracting So...
Hashtag Conversations,Eventgraphs, and User Ego Neighborhoods: Extracting So...
 
Damage Assessment from Social Media Imagery Data During Disasters
Damage Assessment from Social Media Imagery Data During DisastersDamage Assessment from Social Media Imagery Data During Disasters
Damage Assessment from Social Media Imagery Data During Disasters
 
Studying Online Food Consumption and Production Patterns: Recent Trends and C...
Studying Online Food Consumption and Production Patterns: Recent Trends and C...Studying Online Food Consumption and Production Patterns: Recent Trends and C...
Studying Online Food Consumption and Production Patterns: Recent Trends and C...
 
Ethical and Legal Issues in Computational Social Science - Lecture 7 in Intro...
Ethical and Legal Issues in Computational Social Science - Lecture 7 in Intro...Ethical and Legal Issues in Computational Social Science - Lecture 7 in Intro...
Ethical and Legal Issues in Computational Social Science - Lecture 7 in Intro...
 
Image4Act: Online Social Media Image Processing for Disaster Response
Image4Act: Online Social Media Image Processing for Disaster ResponseImage4Act: Online Social Media Image Processing for Disaster Response
Image4Act: Online Social Media Image Processing for Disaster Response
 
Finding media illustrating events
Finding media illustrating eventsFinding media illustrating events
Finding media illustrating events
 
How to Analyze the Privacy of 750000 Smartphone Apps
How to Analyze the Privacy of 750000 Smartphone Apps How to Analyze the Privacy of 750000 Smartphone Apps
How to Analyze the Privacy of 750000 Smartphone Apps
 
A comparison of Lexicon-based approaches for Sentiment Analysis of microblog ...
A comparison of Lexicon-based approaches for Sentiment Analysis of microblog ...A comparison of Lexicon-based approaches for Sentiment Analysis of microblog ...
A comparison of Lexicon-based approaches for Sentiment Analysis of microblog ...
 
CHI2015 - Citizen Science || Zooniverse
CHI2015 - Citizen Science || ZooniverseCHI2015 - Citizen Science || Zooniverse
CHI2015 - Citizen Science || Zooniverse
 
Introduction to Social Network Analysis
Introduction to Social Network AnalysisIntroduction to Social Network Analysis
Introduction to Social Network Analysis
 

Similar to Community-based Crowdsourcing

Social-aware Opportunistic Routing
Social-aware Opportunistic RoutingSocial-aware Opportunistic Routing
Social-aware Opportunistic RoutingWaldir Moreira
 
Smartweek 2014 London: EU FP7 SocIoTal project overview - Michele Nati - Univ...
Smartweek 2014 London: EU FP7 SocIoTal project overview - Michele Nati - Univ...Smartweek 2014 London: EU FP7 SocIoTal project overview - Michele Nati - Univ...
Smartweek 2014 London: EU FP7 SocIoTal project overview - Michele Nati - Univ...MicheleNati
 
Bridging Sensor Data Streams and Human Knowledge
Bridging Sensor Data Streams and Human KnowledgeBridging Sensor Data Streams and Human Knowledge
Bridging Sensor Data Streams and Human KnowledgeMattia Zeni
 
Jisc/UCISA The Digital Student UCISA 2014 presentation
Jisc/UCISA The Digital Student UCISA 2014 presentationJisc/UCISA The Digital Student UCISA 2014 presentation
Jisc/UCISA The Digital Student UCISA 2014 presentationJim Nottingham
 
IRJET- Smart Dustbin using GPS Tracking
IRJET- Smart Dustbin using GPS TrackingIRJET- Smart Dustbin using GPS Tracking
IRJET- Smart Dustbin using GPS TrackingIRJET Journal
 
Mapping the Maker Movement and its impact @ Maker Faire Barcelona 2019
Mapping the Maker Movement and its impact @ Maker Faire Barcelona 2019Mapping the Maker Movement and its impact @ Maker Faire Barcelona 2019
Mapping the Maker Movement and its impact @ Maker Faire Barcelona 2019Massimo Menichinelli
 
User behavior model & recommendation on basis of social networks
User behavior model & recommendation on basis of social networks User behavior model & recommendation on basis of social networks
User behavior model & recommendation on basis of social networks Shah Alam Sabuj
 
ML for blind people.pptx
ML for blind people.pptxML for blind people.pptx
ML for blind people.pptxJAYKUMARKABRA
 
Ii 05. wp8 dti lodz.poland june 2014
Ii 05. wp8 dti lodz.poland june 2014Ii 05. wp8 dti lodz.poland june 2014
Ii 05. wp8 dti lodz.poland june 2014Maciej Szczepańczyk
 
10.MIL 9. Current and Future Trends in Media and Information.pptx
10.MIL 9. Current and Future Trends in Media and Information.pptx10.MIL 9. Current and Future Trends in Media and Information.pptx
10.MIL 9. Current and Future Trends in Media and Information.pptxEdelmarBenosa3
 
ESWC 2015 - EU Networking Session
ESWC 2015 - EU Networking SessionESWC 2015 - EU Networking Session
ESWC 2015 - EU Networking SessionErik Mannens
 
User Involvement in Software Evolution Practice: A Case Study
User Involvement in Software Evolution Practice: A Case StudyUser Involvement in Software Evolution Practice: A Case Study
User Involvement in Software Evolution Practice: A Case StudyDennis Pagano
 
Considering People with Disabilities as Überusers for Eliciting Generalisable...
Considering People with Disabilities as Überusers for Eliciting Generalisable...Considering People with Disabilities as Überusers for Eliciting Generalisable...
Considering People with Disabilities as Überusers for Eliciting Generalisable...Markel Vigo
 
SC7 Hangout 1: Pilot Development for Secure Societies and adopted technologies
SC7 Hangout 1: Pilot Development for Secure Societies and adopted technologiesSC7 Hangout 1: Pilot Development for Secure Societies and adopted technologies
SC7 Hangout 1: Pilot Development for Secure Societies and adopted technologiesBigData_Europe
 
Social Distance Monitoring and Mask Detection Using Deep Learning Techniques
Social Distance Monitoring and Mask Detection Using Deep Learning TechniquesSocial Distance Monitoring and Mask Detection Using Deep Learning Techniques
Social Distance Monitoring and Mask Detection Using Deep Learning TechniquesIRJET Journal
 
Into the Night - Technology for citizen science
Into the Night - Technology for citizen scienceInto the Night - Technology for citizen science
Into the Night - Technology for citizen scienceMuki Haklay
 
UMOBILE: Universal, mobile-centric and opportunistic communications architecture
UMOBILE: Universal, mobile-centric and opportunistic communications architectureUMOBILE: Universal, mobile-centric and opportunistic communications architecture
UMOBILE: Universal, mobile-centric and opportunistic communications architecturePaulo Milheiro Mendes
 
Evaluating Accessibility-in-Use
Evaluating Accessibility-in-UseEvaluating Accessibility-in-Use
Evaluating Accessibility-in-UseMarkel Vigo
 
Development of a lossy online mouse tracking method for capturing user intera...
Development of a lossy online mouse tracking method for capturing user intera...Development of a lossy online mouse tracking method for capturing user intera...
Development of a lossy online mouse tracking method for capturing user intera...Fajar Purnama
 

Similar to Community-based Crowdsourcing (20)

Social-aware Opportunistic Routing
Social-aware Opportunistic RoutingSocial-aware Opportunistic Routing
Social-aware Opportunistic Routing
 
Smartweek 2014 London: EU FP7 SocIoTal project overview - Michele Nati - Univ...
Smartweek 2014 London: EU FP7 SocIoTal project overview - Michele Nati - Univ...Smartweek 2014 London: EU FP7 SocIoTal project overview - Michele Nati - Univ...
Smartweek 2014 London: EU FP7 SocIoTal project overview - Michele Nati - Univ...
 
Bridging Sensor Data Streams and Human Knowledge
Bridging Sensor Data Streams and Human KnowledgeBridging Sensor Data Streams and Human Knowledge
Bridging Sensor Data Streams and Human Knowledge
 
Jisc/UCISA The Digital Student UCISA 2014 presentation
Jisc/UCISA The Digital Student UCISA 2014 presentationJisc/UCISA The Digital Student UCISA 2014 presentation
Jisc/UCISA The Digital Student UCISA 2014 presentation
 
IRJET- Smart Dustbin using GPS Tracking
IRJET- Smart Dustbin using GPS TrackingIRJET- Smart Dustbin using GPS Tracking
IRJET- Smart Dustbin using GPS Tracking
 
Mapping the Maker Movement and its impact @ Maker Faire Barcelona 2019
Mapping the Maker Movement and its impact @ Maker Faire Barcelona 2019Mapping the Maker Movement and its impact @ Maker Faire Barcelona 2019
Mapping the Maker Movement and its impact @ Maker Faire Barcelona 2019
 
User behavior model & recommendation on basis of social networks
User behavior model & recommendation on basis of social networks User behavior model & recommendation on basis of social networks
User behavior model & recommendation on basis of social networks
 
ML for blind people.pptx
ML for blind people.pptxML for blind people.pptx
ML for blind people.pptx
 
Ii 05. wp8 dti lodz.poland june 2014
Ii 05. wp8 dti lodz.poland june 2014Ii 05. wp8 dti lodz.poland june 2014
Ii 05. wp8 dti lodz.poland june 2014
 
10.MIL 9. Current and Future Trends in Media and Information.pptx
10.MIL 9. Current and Future Trends in Media and Information.pptx10.MIL 9. Current and Future Trends in Media and Information.pptx
10.MIL 9. Current and Future Trends in Media and Information.pptx
 
ESWC 2015 - EU Networking Session
ESWC 2015 - EU Networking SessionESWC 2015 - EU Networking Session
ESWC 2015 - EU Networking Session
 
Adopting a User Modeling Approach to Quantify the City
Adopting a User Modeling Approach to Quantify the CityAdopting a User Modeling Approach to Quantify the City
Adopting a User Modeling Approach to Quantify the City
 
User Involvement in Software Evolution Practice: A Case Study
User Involvement in Software Evolution Practice: A Case StudyUser Involvement in Software Evolution Practice: A Case Study
User Involvement in Software Evolution Practice: A Case Study
 
Considering People with Disabilities as Überusers for Eliciting Generalisable...
Considering People with Disabilities as Überusers for Eliciting Generalisable...Considering People with Disabilities as Überusers for Eliciting Generalisable...
Considering People with Disabilities as Überusers for Eliciting Generalisable...
 
SC7 Hangout 1: Pilot Development for Secure Societies and adopted technologies
SC7 Hangout 1: Pilot Development for Secure Societies and adopted technologiesSC7 Hangout 1: Pilot Development for Secure Societies and adopted technologies
SC7 Hangout 1: Pilot Development for Secure Societies and adopted technologies
 
Social Distance Monitoring and Mask Detection Using Deep Learning Techniques
Social Distance Monitoring and Mask Detection Using Deep Learning TechniquesSocial Distance Monitoring and Mask Detection Using Deep Learning Techniques
Social Distance Monitoring and Mask Detection Using Deep Learning Techniques
 
Into the Night - Technology for citizen science
Into the Night - Technology for citizen scienceInto the Night - Technology for citizen science
Into the Night - Technology for citizen science
 
UMOBILE: Universal, mobile-centric and opportunistic communications architecture
UMOBILE: Universal, mobile-centric and opportunistic communications architectureUMOBILE: Universal, mobile-centric and opportunistic communications architecture
UMOBILE: Universal, mobile-centric and opportunistic communications architecture
 
Evaluating Accessibility-in-Use
Evaluating Accessibility-in-UseEvaluating Accessibility-in-Use
Evaluating Accessibility-in-Use
 
Development of a lossy online mouse tracking method for capturing user intera...
Development of a lossy online mouse tracking method for capturing user intera...Development of a lossy online mouse tracking method for capturing user intera...
Development of a lossy online mouse tracking method for capturing user intera...
 

More from Andrea Mauri

A Big Data Analysis Framework for Model-Based Web User Behavior Analytics
A Big Data Analysis Framework for Model-Based Web User Behavior AnalyticsA Big Data Analysis Framework for Model-Based Web User Behavior Analytics
A Big Data Analysis Framework for Model-Based Web User Behavior AnalyticsAndrea Mauri
 
TripleWave: Spreading RDF Streams on the Web
TripleWave: Spreading RDF Streams on the WebTripleWave: Spreading RDF Streams on the Web
TripleWave: Spreading RDF Streams on the WebAndrea Mauri
 
Model Driven Development of Social Media Environmental Monitoring Applications
Model Driven Development of Social Media Environmental Monitoring ApplicationsModel Driven Development of Social Media Environmental Monitoring Applications
Model Driven Development of Social Media Environmental Monitoring ApplicationsAndrea Mauri
 
Methodologies for the Development of Crowd and Social-based applications
Methodologies for the Development of Crowd and Social-based applicationsMethodologies for the Development of Crowd and Social-based applications
Methodologies for the Development of Crowd and Social-based applicationsAndrea Mauri
 
An explorative approach for Crowdsourcing tasks design
An explorative approach for Crowdsourcing tasks design �An explorative approach for Crowdsourcing tasks design �
An explorative approach for Crowdsourcing tasks design Andrea Mauri
 
Community Profiling for Crowdsourcing Queries
Community Profiling for Crowdsourcing QueriesCommunity Profiling for Crowdsourcing Queries
Community Profiling for Crowdsourcing QueriesAndrea Mauri
 

More from Andrea Mauri (6)

A Big Data Analysis Framework for Model-Based Web User Behavior Analytics
A Big Data Analysis Framework for Model-Based Web User Behavior AnalyticsA Big Data Analysis Framework for Model-Based Web User Behavior Analytics
A Big Data Analysis Framework for Model-Based Web User Behavior Analytics
 
TripleWave: Spreading RDF Streams on the Web
TripleWave: Spreading RDF Streams on the WebTripleWave: Spreading RDF Streams on the Web
TripleWave: Spreading RDF Streams on the Web
 
Model Driven Development of Social Media Environmental Monitoring Applications
Model Driven Development of Social Media Environmental Monitoring ApplicationsModel Driven Development of Social Media Environmental Monitoring Applications
Model Driven Development of Social Media Environmental Monitoring Applications
 
Methodologies for the Development of Crowd and Social-based applications
Methodologies for the Development of Crowd and Social-based applicationsMethodologies for the Development of Crowd and Social-based applications
Methodologies for the Development of Crowd and Social-based applications
 
An explorative approach for Crowdsourcing tasks design
An explorative approach for Crowdsourcing tasks design �An explorative approach for Crowdsourcing tasks design �
An explorative approach for Crowdsourcing tasks design
 
Community Profiling for Crowdsourcing Queries
Community Profiling for Crowdsourcing QueriesCommunity Profiling for Crowdsourcing Queries
Community Profiling for Crowdsourcing Queries
 

Recently uploaded

LLMOps with Azure Machine Learning prompt flow
LLMOps with Azure Machine Learning prompt flowLLMOps with Azure Machine Learning prompt flow
LLMOps with Azure Machine Learning prompt flowNaoki (Neo) SATO
 
P1 Inspection Types in Municity 5 Smartsheet
P1 Inspection Types in Municity 5 SmartsheetP1 Inspection Types in Municity 5 Smartsheet
P1 Inspection Types in Municity 5 SmartsheetMatthewTHawley
 
Agile & Scrum, Certified Scrum Master! Crash Course
Agile & Scrum,  Certified Scrum Master! Crash CourseAgile & Scrum,  Certified Scrum Master! Crash Course
Agile & Scrum, Certified Scrum Master! Crash CourseRohan Chandane
 
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and FlinkDBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and FlinkTimothy Spann
 
Joseph Yoder : Being Agile about Architecture
Joseph Yoder : Being Agile about ArchitectureJoseph Yoder : Being Agile about Architecture
Joseph Yoder : Being Agile about ArchitectureHironori Washizaki
 
Implementing Docker Containers with Windows Server 2019
Implementing Docker Containers with Windows Server 2019Implementing Docker Containers with Windows Server 2019
Implementing Docker Containers with Windows Server 2019VICTOR MAESTRE RAMIREZ
 
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ..."Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...ISPMAIndia
 
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...emili denli
 
The Top Outages of 2023: Analyses and Takeaways
The Top Outages of 2023: Analyses and TakeawaysThe Top Outages of 2023: Analyses and Takeaways
The Top Outages of 2023: Analyses and TakeawaysThousandEyes
 
killingcamp longest common subsequence.pdf
killingcamp longest common subsequence.pdfkillingcamp longest common subsequence.pdf
killingcamp longest common subsequence.pdfssuser82c38d
 
SPM 2024 – Overview of and benefits of AI in Product Management
SPM 2024 – Overview of and benefits of AI in Product ManagementSPM 2024 – Overview of and benefits of AI in Product Management
SPM 2024 – Overview of and benefits of AI in Product ManagementISPMAIndia
 
killing camp week 6 problem - maximal matrix.pdf
killing camp week 6 problem - maximal matrix.pdfkilling camp week 6 problem - maximal matrix.pdf
killing camp week 6 problem - maximal matrix.pdfssuser82c38d
 
Automation for Bonterra Impact Management (fka Apricot)
Automation for Bonterra Impact Management (fka Apricot)Automation for Bonterra Impact Management (fka Apricot)
Automation for Bonterra Impact Management (fka Apricot)Jeffrey Haguewood
 
No more Dockerfiles? Buildpacks to help you ship your image!
No more Dockerfiles? Buildpacks to help you ship your image!No more Dockerfiles? Buildpacks to help you ship your image!
No more Dockerfiles? Buildpacks to help you ship your image!Anthony Dahanne
 
killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이
killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이
killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이ssuser82c38d
 
Orion Context Broker introduction 20240227
Orion Context Broker introduction 20240227Orion Context Broker introduction 20240227
Orion Context Broker introduction 20240227Fermin Galan
 
Machine Learning Basics for Dummies (no math!)
Machine Learning Basics for Dummies (no math!)Machine Learning Basics for Dummies (no math!)
Machine Learning Basics for Dummies (no math!)Dmitry Zinoviev
 
Cybersecurity Measures For Remote Workers.pdf
Cybersecurity Measures For Remote Workers.pdfCybersecurity Measures For Remote Workers.pdf
Cybersecurity Measures For Remote Workers.pdfCIOWomenMagazine
 
Welcome to AltTask - the nexus where innovation converges with empowerment!
Welcome to AltTask - the nexus where innovation converges with empowerment!Welcome to AltTask - the nexus where innovation converges with empowerment!
Welcome to AltTask - the nexus where innovation converges with empowerment!alttaskcom
 

Recently uploaded (20)

LLMOps with Azure Machine Learning prompt flow
LLMOps with Azure Machine Learning prompt flowLLMOps with Azure Machine Learning prompt flow
LLMOps with Azure Machine Learning prompt flow
 
P1 Inspection Types in Municity 5 Smartsheet
P1 Inspection Types in Municity 5 SmartsheetP1 Inspection Types in Municity 5 Smartsheet
P1 Inspection Types in Municity 5 Smartsheet
 
Agile & Scrum, Certified Scrum Master! Crash Course
Agile & Scrum,  Certified Scrum Master! Crash CourseAgile & Scrum,  Certified Scrum Master! Crash Course
Agile & Scrum, Certified Scrum Master! Crash Course
 
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and FlinkDBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
 
eLearning Content Development Company Code and Pixels.pdf
eLearning Content Development Company Code and Pixels.pdfeLearning Content Development Company Code and Pixels.pdf
eLearning Content Development Company Code and Pixels.pdf
 
Joseph Yoder : Being Agile about Architecture
Joseph Yoder : Being Agile about ArchitectureJoseph Yoder : Being Agile about Architecture
Joseph Yoder : Being Agile about Architecture
 
Implementing Docker Containers with Windows Server 2019
Implementing Docker Containers with Windows Server 2019Implementing Docker Containers with Windows Server 2019
Implementing Docker Containers with Windows Server 2019
 
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ..."Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
 
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...
 
The Top Outages of 2023: Analyses and Takeaways
The Top Outages of 2023: Analyses and TakeawaysThe Top Outages of 2023: Analyses and Takeaways
The Top Outages of 2023: Analyses and Takeaways
 
killingcamp longest common subsequence.pdf
killingcamp longest common subsequence.pdfkillingcamp longest common subsequence.pdf
killingcamp longest common subsequence.pdf
 
SPM 2024 – Overview of and benefits of AI in Product Management
SPM 2024 – Overview of and benefits of AI in Product ManagementSPM 2024 – Overview of and benefits of AI in Product Management
SPM 2024 – Overview of and benefits of AI in Product Management
 
killing camp week 6 problem - maximal matrix.pdf
killing camp week 6 problem - maximal matrix.pdfkilling camp week 6 problem - maximal matrix.pdf
killing camp week 6 problem - maximal matrix.pdf
 
Automation for Bonterra Impact Management (fka Apricot)
Automation for Bonterra Impact Management (fka Apricot)Automation for Bonterra Impact Management (fka Apricot)
Automation for Bonterra Impact Management (fka Apricot)
 
No more Dockerfiles? Buildpacks to help you ship your image!
No more Dockerfiles? Buildpacks to help you ship your image!No more Dockerfiles? Buildpacks to help you ship your image!
No more Dockerfiles? Buildpacks to help you ship your image!
 
killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이
killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이
killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이
 
Orion Context Broker introduction 20240227
Orion Context Broker introduction 20240227Orion Context Broker introduction 20240227
Orion Context Broker introduction 20240227
 
Machine Learning Basics for Dummies (no math!)
Machine Learning Basics for Dummies (no math!)Machine Learning Basics for Dummies (no math!)
Machine Learning Basics for Dummies (no math!)
 
Cybersecurity Measures For Remote Workers.pdf
Cybersecurity Measures For Remote Workers.pdfCybersecurity Measures For Remote Workers.pdf
Cybersecurity Measures For Remote Workers.pdf
 
Welcome to AltTask - the nexus where innovation converges with empowerment!
Welcome to AltTask - the nexus where innovation converges with empowerment!Welcome to AltTask - the nexus where innovation converges with empowerment!
Welcome to AltTask - the nexus where innovation converges with empowerment!
 

Community-based Crowdsourcing

  • 1. Dipartimento di Elettronica, Informazione e Bioingegneria COMMUNITY-BASED CROWDSOURCING Marco Brambilla, Stefano Ceri, Andrea Mauri, Riccardo Volonterio
  • 2. Dipartimento di Elettronica, Informazione e Bioingegneria Crowd-based Applications  Emerging crowd-based applications: – opinion mining – localized information gathering – marketing campaigns – expert response gathering  General structure: – the requestor poses some questions – a wide set of responders are in charge of providing answers (typically unknown to the requestor) – the system organizes a response collection campaign SOCM’14, Monday, April 7 Community-basedCrowdsourcing 3
  • 3. Dipartimento di Elettronica, Informazione e Bioingegneria Our approach  Combines a conceptual framework, a specification paradigm and a reactive execution control environment  Supports designing, deploying, and monitoring applications on top of crowd-based systems – Design is top-down, platform-independent – Deployment turns declarative specifications into platform- specific implementations which include social networks and crowdsourcing platforms – Monitoring provides reactive control, which guarantees applications’ adaptation and interoperability SOCM’14, Monday, April 7 Community-basedCrowdsourcing 4
  • 4. Dipartimento di Elettronica, Informazione e Bioingegneria Control  Controlling crowdsourcing tasks is a fundamental issue – Cost – Time – Quality  In our approach – Implemented trough active rules • Control is easily expressible  Simple control data structures  Familiar formalism • Support to arbitrarily complex controls  Extensibility mechanisms • Active rules can be system-generated  Well-defined semantic SOCM’14, Monday, April 7 Community-basedCrowdsourcing 5
  • 5. Dipartimento di Elettronica, Informazione e Bioingegneria Community Set of people that share  Interests  Feature ..or belong to  common entity  social network SOCM’14, Monday, April 7 Community-basedCrowdsourcing 6
  • 6. Dipartimento di Elettronica, Informazione e Bioingegneria Leveraging communities  Why? – Experts – More engaged  How? – Determine the communities of performers – Monitor them taking into the account the behavior of their member • Community Control SOCM’14, Monday, April 7 Community-basedCrowdsourcing 7
  • 7. Dipartimento di Elettronica, Informazione e Bioingegneria Community Control Community control allow the adaptation of the crowdsourcing campaign  Task / Object allocation  Static / Dynamic Implemented with the reactive environment present in our approach SOCM’14, Monday, April 7 Community-basedCrowdsourcing 8
  • 8. Dipartimento di Elettronica, Informazione e Bioingegneria Example (dynamic control) µTObjExecution Performer TaskImage StatusStartTsEndTs µTaskID Object Control Performer Control Task Control C om pO bjs TaskIDCom pExecs Name PerformerID Score ProfPhoto M aterials NonRelevant PeoplePlace TaskID ObjectID Correct ImgUrl Category ProfessorID ObjectID Answer Eval ObjectID PerformerID TaskID PerformerID CommunityCommunityID Name Community Control CommunityID Score CommunityID Enabled Status Status LastExec Tests Execs SOCM’14, Monday, April 7 Community-basedCrowdsourcing 10 e: AFTER UPDATE FOR μTObjExecution c: CommunityControl[CommunityID== NEW.CommunityID].score<=0.5 CommunityControl[CommunityID== NEW.CommunityID].eval=10 a: SET CommunityControl[CommunityID == DB-Group].Enabled = true ?
  • 9. Dipartimento di Elettronica, Informazione e Bioingegneria Crowdsearcher A prototype that allows the definition, execution and control of a crowdsourcing campaign according to our approach SOCM’14, Monday, April 7 Community-basedCrowdsourcing 11 http://crowdsearcher.search-computing.org/
  • 10. Dipartimento di Elettronica, Informazione e Bioingegneria Experiment  16 professors within two research groups in our department (DB and AI groups)  The top 50 images returned by the Google Image API for each query  Each microtask consisted of evaluating 5 images regarding a professor.  Results are accepted (and thus the corresponding object is closed) when enough agreement on the class of the image is reached  Closed objects are removed from new executions. SOCM’14, Monday, April 7 Community-basedCrowdsourcing 12
  • 11. Dipartimento di Elettronica, Informazione e Bioingegneria Communities The communities:  the research group of the professor,  the research area containing the group (e.g. Computer Science)  and the whole department (which accounts for more than 600 people in different areas) Invitations are sent:  inside-out: we started with invitations to experts, e.g. people the same groups as the professor (DB and AI), and then expanded invitations to Computer Science, then to the whole Department, and finally to open social networks (Alumni and PhDs communities on Facebook and Linkedin);  outside-in: we proceeded in the opposite way, starting with the Department members, then restricting to Computer Scientists, and finally to the group's members. SOCM’14, Monday, April 7 Community-basedCrowdsourcing 13
  • 12. Dipartimento di Elettronica, Informazione e Bioingegneria Number of performers per community SOCM’14, Monday, April 7 Community-basedCrowdsourcing 14 0" 10" 20" 30" 40" 50" 60" 70" 7/18/2013" 7/19/2013" 7/20/2013" 7/21/2013" 7/22/2013" 7/23/2013" 7/24/2013" 7/25/2013" 7/26/2013" 7/27/2013" 7/28/2013" #"Performers" Time" research"group" research"area" department" social"network" total" 46% 24% 16% 9 / “a lot”
  • 13. Dipartimento di Elettronica, Informazione e Bioingegneria Precision of performers per community SOCM’14, Monday, April 7 Community-basedCrowdsourcing 15 0" 0.1" 0.2" 0.3" 0.4" 0.5" 0.6" 0.7" 0.8" 0.9" 1" 0" 500" 1000" 1500" 2000" 2500" 3000" Precision) #Evalua0ons) research"group" research"area" department" social"network" total"
  • 14. Dipartimento di Elettronica, Informazione e Bioingegneria Precision of the evaluated objects  Precision decreases with less expert communities  Inside-out strategy (from expert to generic users) outperforms Outside-in strategy (from generic to expert users) SOCM’14, Monday, April 7 Community-basedCrowdsourcing 16 0.6$ 0.65$ 0.7$ 0.75$ 0.8$ 0.85$ 0.9$ 0.95$ 1$ 0$ 100$ 200$ 300$ 400$ 500$ 600$ 700$ 800$ Precision) #Closed)Objects) precision$(main$experiment)$ precision$(reverse$invita<ons)$
  • 15. Dipartimento di Elettronica, Informazione e Bioingegneria General observations Experts from community feel more engaged with the task  They are more demanding with respect to the quality of the application UI and the evaluated objects  Provide feedbacks on the application, question and the objects evaluated – “How is it possible that this image is related to me?!” SOCM’14, Monday, April 7 Community-basedCrowdsourcing 17
  • 16. Dipartimento di Elettronica, Informazione e Bioingegneria Thanks for you attention Any question? SOCM’14, Monday, April 7 Community-basedCrowdsourcing 19