SLUA: Towards Semantic Linking of Users with Actions in Crowdsourcing


Published on

Recent advances in web technologies allow people to help solve complex problems by performing online tasks in return for money, learning, or fun. At present, human contribution is limited to the tasks defined on individual crowdsourcing platforms. Furthermore, there is a lack of tools and technologies that support matching of tasks with appropriate users, across multiple systems. A more explicit capture of the semantics of crowdsourcing tasks could enable the design and development of matchmaking services between users and tasks. The paper presents the SLUA ontology that aims to model users and tasks in crowdsourcing systems in terms of the relevant actions, capabilities, and rewards. This model describes different types of human tasks that help in solving complex problems using crowds. The paper provides examples of describing users and tasks in some real world systems, with SLUA ontology.

Published in: Technology, Education
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

SLUA: Towards Semantic Linking of Users with Actions in Crowdsourcing

  1. 1. SLUA:  TOWARDS  SEMANTIC  LINKING  OF   USERS  WITH  ACTIONS  IN  CROWDSOURCING   www.insight-­‐   Umair  ul  Hassan,  Sean  O’Riain,  Edward  Curry   INSIGHT  Centre  for  Data  Analy4cs   Na4onal  University  of  Ireland,  Galway     1st International Workshop on Crowdsourcing the Semantic Web, CrowdSem’13, Sydney, Australia
  2. 2. Paper  Overview   www.insight-­‐   •  MoJvaJon   –  MulJple  crowdsourcing  plaBorms     –  Lack  of  tools  for  finding  tasks   –  Need  to  query  across  plaPorms  for  skills  and  knowledge  of  workers   •  Problem   –  Enabling  interoperability  across  crowdsourcing  plaBorms   –  Suppor4ng  users  in  their  search  for  tasks   –  Enable  task  and  user  matching  services   •  ContribuJon   –  An  ini4al  lightweight  ontology  for  describing  crowd  sourced  tasks  and   users  with  regard  to  human  capabili4es,  ac4ons,  and  rewards   2
  3. 3. Agenda   www.insight-­‐   •  Mo4va4on   –  Crowdsourcing  Landscape   –  Seman4c  Heterogeneity   •  •  •  •  Challenges   SLUA  Ontology   Examples   Summary   3
  4. 4. Crowdsourcing  Landscape   www.insight-­‐   4
  5. 5. Crowdsourcing  Landscape   www.insight-­‐   5
  6. 6. Challenges   www.insight-­‐   •  Difficult  to  interoperate  across  crowdsourcing   systems  and  plaPorms   –  e.g.  searching  for  appropriate  workers  on   StackExchange  for  Wiki  edi4ng  tasks   •  VariaJons  of  data  semanJcs  across  systems  and   plaPorms   –  Different  APIs  used  by  current  marketplaces   •  ExisJng  Taxonomies  plaPorm-­‐centric   –  Categorize  crowdsourcing  plaBorms  instead  of  tasks   –  LiPkle  considera4on  human  of  factors  such  as  ac4ons,   capabili4es,  mo4va4on   6
  7. 7. Heterogeneous  Crowds   www.insight-­‐   Tasks Platforms Workers Collaborative Data Curation Cyber Physical Social System •  Mul4ple  requesters,  tasks,  workers,  plaBorm   7
  8. 8. PlaBorm  Heterogeneity   www.insight-­‐   Tasks   Human  AcJons   Required   CapabiliJes   Rewards   Wikipedia   Create  Content   Edit  Content   Moderate  Content   Write  text   Include  references   Highlight  mistakes   Domain  Knowledge   Wri4ng   Research   Social  Good   Quora   Ask  Ques4ons   Answer  Ques4ons   Write  text   Domain  Knowledge     Reputa4on   Amazon   Mechanical   Turk   Micro  Tasks   Transcribe,   Translate,   Categorize,  etc.   Various  Capabili4es   Money   TaskRabbit   Physical  Tasks   Collect  Item   Deliver  Item   Shop  Item,  etc.   Various  Capabili4es   Money   Microtask   Form  Filling   Scan  Correc4on   Data  Verifica4on   Play  games   Online  gaming   Fun   8
  9. 9. Proposed  Solu4on   www.insight-­‐   •  A  common  model  for  describing  tasks  in   crowdsourcing   •  Seman4cally  Linked  Users  and  Ac4ons  (SLUA)   –  Lightweight  semanJc  descripJon  of  crowdsourcing   tasks  in  terms  of  human  capabiliJes,  acJons,  and   rewards   •  SLUA   –  Gaelic  word  for…                              …“Crowd”     9
  10. 10. SLUA  Design  Medthdology   www.insight-­‐   1.  Enumerate  similar  terms  on  crowdsourcing   plaBorms   2.  Define  the  main  concept  in  each  group  of   terms   3.  Compare  with  exis4ng  ontologies   4.  Define  core  classes  and  their  rela4onships   5.  Extend  core  classes  with  subclasses   6.  Create  example  instances   10
  11. 11. Crowdsourcing  Terminology   www.insight-­‐   Amazon   Mechanical   Turk Mobileworks ShorXask CrowdFlower Task HIT Task ShortTask Microtask User Worker   Requester Worker   Developer Solver   Seeker Contributor   Customer Capability Qualifica4on Filter     Reward Payment Payment Reward Payment •  Terms  used  for  similar  concepts  in  crowdsourcing  plaBorms   11
  12. 12. Conceptual  Requirements   www.insight-­‐   •  Task:  A  unit  of  work  to  be  performed  by  people  in  the   crowd   •  AcJon:  The  cogni4ve  or  psychomotor  ac4vity  that   leads  towards  the  comple4on  of  a  task   •  User:  The  human  par4cipant,  commonly  described  as   “worker”  in  crowdsourcing  marketplaces.   •  Capability:  The  human  ability,  knowledge,  or  skill  that   allows  a  user  to  perform  the  necessary  ac4ons  for  task   comple4on.   •  Reward:  A  core  concept  to  the  mo4va4on  of  people  in   the  crowd   12
  13. 13. Exis4ng  Ontologies     www.insight-­‐   Concept PIMO TMO HRM-­‐O FOAF SIOC Task Task Task       AcJon           User Person   Job  Seeker Person UserAccount Reward     Compensa4on     Reputa4on           Money     Salary     Fun           Altruism           Learning           Capability           Loca4on     Loca4on     Skill   Skill       Knowledge           Ability     Ability     Availability     Interval     Personal Information Management Ontology (PIMO) Task Management Ontology (TMO) Human Resource Management Ontology (HRM-O) Friend of a Friend (FOAF) Semantically Interlinked Online Communities (SIOC) 13
  14. 14. SLUA  Core  Classes  and  Proper4es   www.insight-­‐   earns User Reward performs possesses Action Capability offers includes Task requires 14
  15. 15. SLUA  Sub-­‐classes   www.insight-­‐   •  Capability   –  The  ability  of  people  to  do  things  -­‐  both  the  capacity   and  the  opportuni4es  to  do  things.   –  Main  capabili4es  in  literature   •  Knowledge,  Skill,  Ability,  and  Others  (e.g.  Loca4on,   Availability)   •  Reward   –  The  benefit  generated  from  the  use  of  capability  in   both  labour  market  and  non-­‐labour  market  ac4vi4es.   –  Main  rewards  in  literature   •  Reputa4on,  Money,  Fun,  Learning,  Altruism  or  Social  Good   15
  16. 16. SLUA  Ontology   www.insight-­‐   Reputation Money Fun Altruism Learning subClassOf earns User Reward performs possesses Action Capability offers includes Task requires subClassOf Location Skill Knowledge 16 Ability Availability
  17. 17. Example  Task   www.insight-­‐   Please  consider  adding  full  cita4ons  to   the  Wikipedia  ar4cle   slua:Task a slua:Loca4on a hPp:// A3_road/tasks/1 slua:requires slua:Reward :loc1 hPp:// resource/London slua:Knowledge rdfs:label slua:requires slua:includes :ac1 a a :knw1 17 a :rw1 rdfs:label hPp:// resource/London slua:Ac4on slua:offers Wiki  page  edit a slua:Reputa4on
  18. 18. SLUA  in  Ac4on   www.insight-­‐   Tasks SLUA Mediated Task Routing Task Modelling Task Routing Worker Profiling Matching Collaborative Data Curation Capability Capability Models Task Model Model Cold Start Worker Worker Profiles Worker Profiles Profiless Ordering SLUA Mediated Infrastructure Services Cyber Physical Social System Application Interface, User Interface, Identity Management, Notification Services 18 Workers
  19. 19. Collabora4ve  Data  Cura4on   www.insight-­‐   Programmers Managers Web of Data DQ Rules & Algorithms Human Computation Entity Linking Data Fusion Relation Extraction Databases External Crowd - High Availability - Large Scale - Expertise Variety Relevance Judgment Data Verification Disambiguation Textual Content Clean Data 19 Internal Community - Domain Knowledge - High Quality Responses - Trustable
  20. 20. Cyber-­‐Physical  Social  Systems   www.insight-­‐   Sensor Data Collection Temperature of Room 202e Energy Decision Models Human Actuation Close Window Room 202e Monitored Physical Environment Located near Room 202e 20
  21. 21. Summary  &  Future  Work   www.insight-­‐   •  SLUA  is  an  ini4al  step  towards  defining  a  light-­‐ weight  ontology  for  describing  tasks,  acJons,   users,  rewards,  and  capabiliJes  in  crowdsourcing   plaBorms   •  Future  Plans   –  Enumerate  design  with  addi4onal  crowdsourcing   plaBorms   –  Prototype  SLUA-­‐based  for  cross  plaBorm  query   –  Task  rou4ng  system  used  match  between  tasks  and   users  with  SLUA  descriptors   21
  22. 22. Further  Reading   www.insight-­‐               International Workshop on “Crowdsourcing the Semantic Web”  Sydney, 21 October 2013 Ontology  Available  at:  hXp://         U.  Ul  Hassan,  S.  O’Riain,  and  E.  Curry,  “SLUA:  Towards  SemanJc  Linking  of  Users  with  AcJons  in   Crowdsourcing,”  in  1st  InternaJonal  Workshop  on  Crowdsourcing  the  SemanJc  Web,  2013.       hXp://­‐ul-­‐hassan     22